Stop by and check out the following events related to Darshan at SC18:

• Glenn Lockwood et al. “A Year in the Life of a Parallel File System”, SC18 technical program paper.
• Robert Latham et al. “Parallel I/O in Practice”, Tutorial.
• Jakob Luettgau et al. “Toward Understanding I/O Behavior in HPC Workflows”, PDSW workshop paper.
• Philip Carns et al. “Analyzing Parallel I/O”, Birds of a Feather session.

The R&D 100 Conference recently announced that Darshan is a 2018 R&D 100 Award finalist.  The award winners will be announced in November 2018.  Thank you everyone for your support!

Update: the normal Darshan download links are now working again.
Our FTP server is down as of 03/29/2018 and we are not yet sure when it will be returned to service.  If you need a tar ball with the latest Darshan release, but our normal download links aren’t working, then you can use this link instead:
https://xgitlab.cels.anl.gov/darshan/darshan/repository/darshan-3.1.6/archive.tar.gz
That will generate a tar ball directly from the git repository tag.

Darshan version 3.1.6 is available for download here. This release addresses two bugs discovered in version 3.1.5:

• Bug fix in mapping of Darshan’s ‘MPI_File_read_all’ wrapper to the underlying MPI symbol. This results in a crash if a dynamically-linked application preloads the Darshan library and calls ‘MPI_File_read_all’.
• Bug fix for linker failures caused when an application links in a library that calls MPI externally (e.g., PETSc).
  • Reported by Bilel Hadri

We highly recommend upgrading from version 3.1.5 to 3.1.6 particularly for deployments on production systems, as these bugs could be disruptive for end users.
Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

Darshan version 3.1.5 is now available for download here.
The list of new features and bug fixes include:

• Darshan’s MPI instrumentation method now intercepts both MPI and PMPI symbols to workaround MPI library bindings that are calling PMPI routines directly
  • Contributed in part by Chris Zimmer
• darshan-util now includes an analysis tool for visualizing DXT traces: dxt_analyzer
  • ​Contributed by Alex Sim
• Darshan now supports the DARSHAN_EXCLUDE_DIRS environment variable, which specifies a comma-separated list of paths to avoid instrumenting at runtime
  • Contributed by Cristian Simarro
• Darshan’s regression test system now includes test harnesses for Cray systems @ the ALCF (Theta) and NERSC (Cori/Edison)
• Bug fix in Darshan’s shutdown procedure leading to potentially negative timer values
• Bug fix in Darshan’s autoconf scripts causing specified libbz2 locations to be ignored
  • Contributed by Glenn Lockwood
• Bug fix in Darshan’s wrapping of H5get_libversion that was causing linker failures in some cases
  • Reported by Jialin Liu
• Bug fix in darshan-merge utility related to logs containing DXT data
  • Reported by Glenn Lockwood

Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

We encourage anyone interested in Darshan and related technology to attend the following events at SC17 in Denver:

• Monday’s “Parallel I/O in Practice” tutorial by Rob Latham, Rob Ross, Brent Welch, and Katie Antypas.  It covers a broad range of topics in parallel I/O, including how to characterize and understand I/O behavior:
  • http://sc17.supercomputing.org/presentation/?id=tut145&sess=sess231
• Monday’s PDSW-DISCS workshop will include the presentation of a paper entitled “UMAMI: A Recipe for Generating Meaningful Metrics through Holistic I/O Performance Analysis” by Glenn Lockwood.  It highlights how to integrate I/O monitoring data from many sources into a coherent view of I/O behavior:
  • http://www.pdsw.org/index.shtml
• Thursday’s “Analyzing Parallel I/O” BoF by Phil Carns and Julian Kunkel.  Several guests will share their experiences in analyzing I/O performance and participate in a panel discussion:
  • http://sc17.supercomputing.org/presentation/?id=bof136&sess=sess364
  • https://wr.informatik.uni-hamburg.de/events/2017/bof-analyzing

A new  pre-release version (3.1.5-pre1) of Darshan is available for download here. Please be aware that this pre-release is experimental and not recommended for full-time use in production yet. An official 3.1.5 release will be available in a few weeks.
The list of new features and bug fixes include:

• Darshan’s MPI instrumentation method now intercepts both MPI and PMPI symbols to workaround MPI library bindings that are calling PMPI routines directly
  • Contributed in part by Chris Zimmer
• darshan-util now includes an analysis tool for visualizing DXT traces: dxt_analyzer
  • Contributed by Alex Sim
• Darshan now supports the DARSHAN_EXCLUDE_DIRS environment variable, which specifies a comma-separated list of paths to avoid instrumenting at runtime
  • Contributed by Cristian Simarro
• Darshan’s regression test system now includes test harnesses for Cray systems @ the ALCF (Theta) and NERSC (Cori/Edison)
• Bug fix in Darshan’s shutdown procedure leading to potentially negative timer values
• Bug fix in Darshan’s autoconf scripts causing specified libbz2 locations to be ignored
  • Contributed by Glenn Lockwood

Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

A new version of Darshan (3.1.4) is available for download here. This release addresses a couple of minor bugs in the previous Darshan release:

• darshan-parser causing segmentation fault when parsing logs with DXT module data present
• darshan-diff utility not producing output

Documentation for this version of Darshan can be found here.
Please let us know if you have any issues or any feedback you would like to share with us, either using the Darshan-users mailing list or the Darshan GitLab page.

The official release for Darshan version 3.1.3 is now available for download here. We have tested this release extensively and believe it is stable and ready for use in production. On top of the numerous new features and bug fixes included in the darshan-3.1.3-pre1 release, the official version has one additional bug fix:

• Instrumentation of mmap calls is now disabled for dynamically linked applications (i.e., applications that LD_PRELOAD darshan)
  • this avoids potential deadlock issues on Cray systems when using dynamically linked executables (reported by Cristian Simarro)

You can find updated documentation for darshan-runtime and darshan-util components here.
Please let us know if you have any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

A new Darshan pre-release (3.1.3-pre1) is now available for download here. Please be aware that this pre-release is experimental and not recommended for full-time use in production yet. We hope to have the official 3.1.3 release ready some time next week.
Here is the list of new features and bug fixes in this release:

• New DxT (Darshan eXtended Tracing) instrumentation modules that provide fine-grained tracing data of read/write operations at the POSIX & MPI-IO layers
  • This tracing module is not enabled by default, and it is envisioned that it be enabled selectively at runtime for applications of interest
  • This code was contributed by Cong Xu and Intel’s High Performance Data Division (HPDD)
• Darshan now captures command line arguments for Fortran applications
  • This fix was contributed by Cristian Simarro
• Fix potential deadlock when using the Cray hugepages module and Darshan
  • Reported by Glenn Lockwood & Cristian Simarro
• Fix potential segmentation fault when Darshan exceeds the maximum number of tracked access sizes/strides
  • Reported by Glenn Lockwood
• Disable Darshan’s HDF5 module by default to avoid potential ABI compatibility issues with HDF5
  • New configure switches are provided to enable Darshan support for specific HDF5 library versions
  • Problem report and initial patch provided by Karl-Ulrich Bamberg
• Add whitelist to Darshan’s list of file path exclusions to prevent I/O to/from Cray’s Datawarp service from being filtered out
  • Contributed by Glenn Lockwood

Updated documentation related to using the DxT tracing modules can be found in the source tree (‘darshan-runtime/docs’ for the Darshan runtime library & ‘darshan-util/docs’ for Darshan’s utilities). To summarize, DxT tracing can be enabled at runtime by exporting the ‘DXT_ENABLE_IO_TRACE’ environment variable. DxT also has a corresponding parser utility named ‘darshan-dxt-parser’ that can be used to parse trace data out of Darshan log files.
As always, let us know if you have any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.