Conduit

Conduit: Simplified Data Exchange for HPC Simulations

Introduction

Conduit is an open source project from Lawrence Livermore National Laboratory that provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks.

Conduit’s Core API provides:

  • A flexible way to describe hierarchal data:

    A JSON-inspired data model for describing hierarchical in-core scientific data.

  • A sane API to access hierarchal data:

    A dynamic API for rapid construction and consumption of hierarchical objects.

Conduit is under active development and targets Linux, OSX, and Windows platforms. The C++ API underpins the other language APIs and currently has the most features. We are still filling out the C, Fortran, and Python APIs.

For more background, please see Presentations.

Unique Features

Conduit was built around the concept that an intuitive in-core data description capability simplifies many other common tasks in the HPC simulation eco-system. To this aim, Conduit’s Core API:

  • Provides a runtime focused in-core data description API that does not require repacking or code generation.
  • Supports a mix of externally owned and Conduit allocated memory semantics.

Conduit Project Resources

Online Documentation

http://software.llnl.gov/conduit/

Github Source Repo

https://github.com/llnl/conduit

Issue Tracker

https://github.com/llnl/conduit/issues

LLNL Collaboration Zone Bitbucket Server Source Repo (Mirror for LLNL Users)

https://lc.llnl.gov/bitbucket/projects/CON/repos/conduit/browse

Conduit Libraries

The conduit library provides Conduit’s core data API. The relay and blueprint libraries provide higher-level services built on top of the core API.

conduit

  • Provides Conduit’s Core API in C++ and subsets of Core API in Python, C, and Fortran.
  • Optionally depends on Fortran and Python with NumPy

relay

  • Provides:
    • I/O functionally beyond simple binary, memory mapped, and json-based text file I/O.
    • A light-weight web server for REST and WebSocket clients.
    • Interfaces for MPI communication using conduit::Node instances as payloads.
  • Optionally depends on silo, hdf5, szip and mpi

blueprint

  • Provides interfaces for common higher-level conventions and data exchange protocols (eg. describing a “mesh”) using Conduit.
  • No optional dependancies

See the User Documentation for more details on these libraries.

Contributors

  • Cyrus Harrison (LLNL)
  • Brian Ryujin (LLNL)
  • Adam Kunen (LLNL)
  • Kathleen Biagas (LLNL)
  • Eric Brugger (LLNL)
  • Aaron Black (LLNL)
  • George Zagaris (LLNL)
  • Kenny Weiss (LLNL)
  • Matt Larsen (LLNL)
  • Joe Ciurej (LLNL)
  • George Aspesi (Harvey Mudd)
  • Justin Bai (Harvey Mudd)
  • Rupert Deese (Harvey Mudd)
  • Linnea Shin (Harvey Mudd)

In 2014 and 2015 LLNL sponsored a Harvey Mudd Computer Science Clinic project focused on using Conduit in HPC Proxy apps. You can read about more details about the clinic project from this LLNL article: http://computation.llnl.gov/newsroom/hpc-partnership-harvey-mudd-college-and-livermore

Indices and tables