Reading YAML and JSON Strings¶

Parsing text with Node::parse()¶

Node::parse() parses YAML and JSON strings into a Node tree.

std::string yaml_txt("mykey: 42.0");

Node n;
n.parse(yaml_txt,"yaml");

std::cout << n["mykey"].as_float64() <<std::endl;

n.print_detailed();

42

{
  "mykey": 
  {
    "dtype":"float64",
    "number_of_elements": 1,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": 42.0
  }
}

std::string json_txt("{\"mykey\": 42.0}");

Node n;
n.parse(json_txt,"json");

std::cout << n["mykey"].as_float64() <<std::endl;

n.print_detailed();

42

{
  "mykey": 
  {
    "dtype":"float64",
    "number_of_elements": 1,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": 42.0
  }
}

The first argument is the string to parse and the second argument selects the protocol to use when parsing.

Valid Protocols: json, conduit_json, conduit_base64_json, yaml.

json and yaml protocols parse pure JSON or YAML strings. For leaf nodes wide types such as int64, uint64, and float64 are inferred.

Homogeneous numeric lists are parsed as Conduit arrays.

std::string yaml_txt("myarray: [0.0, 10.0, 20.0, 30.0]");

Node n;
n.parse(yaml_txt,"yaml");

n["myarray"].print();

n.print_detailed();

[0.0, 10.0, 20.0, 30.0]

{
  "myarray": 
  {
    "dtype":"float64",
    "number_of_elements": 4,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": [0.0, 10.0, 20.0, 30.0]
  }
}

conduit_json parses JSON with conduit data type information, allowing you to specify bitwidth style types, strides, etc.
conduit_base64_json combines the conduit_json protocol with an embedded base64-encoded data block

Generators¶

Using Generator instances¶

Node::parse() is sufficient for most use cases, but you can also use a Generator instance to parse JSON and YAML. Additionally, Generators can parse a conduit JSON schema and bind it to in-core data.

Generator g("{test: {dtype: float64, value: 100.0}}","conduit_json");

Node n;
g.walk(n);

std::cout << n["test"].as_float64() <<std::endl;
n.print();
n.print_detailed();

100

test: 100.0


{
  "test": 
  {
    "dtype":"float64",
    "number_of_elements": 1,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": 100.0
  }
}

Like Node::parse(), Generators can also parse pure JSON or YAML. For leaf nodes: wide types such as int64, uint64, and float64 are inferred.

Generator g("{test: 100.0}","json");

Node n;
g.walk(n);
    
std::cout << n["test"].as_float64() <<std::endl;
n.print_detailed();
n.print();

100

{
  "test": 
  {
    "dtype":"float64",
    "number_of_elements": 1,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": 100.0
  }
}

test: 100.0

Generator g("test: 100.0","yaml");

Node n;
g.walk(n);
    
std::cout << n["test"].as_float64() <<std::endl;
n.print_detailed();
n.print();

100

{
  "test": 
  {
    "dtype":"float64",
    "number_of_elements": 1,
    "offset": 0,
    "stride": 8,
    "element_bytes": 8,
    "endianness": "little",
    "value": 100.0
  }
}

test: 100.0

Schemas can be bound to in-core data.

float64 vals[2];
Generator g("{a: {dtype: float64, value: 100.0}, b: {dtype: float64, value: 200.0} }",
            "conduit_json",
            vals);

Node n;
g.walk_external(n);

std::cout << n["a"].as_float64() << " vs " << vals[0] << std::endl;
std::cout << n["b"].as_float64() << " vs " << vals[1] << std::endl;

n.print();

Node ninfo;
n.info(ninfo);
ninfo.print();

100 vs 100
200 vs 200

a: 100.0
b: 200.0


mem_spaces: 
  0x7ffee7d56140: 
    path: "a"
    type: "external"
total_bytes_allocated: 0
total_bytes_mmaped: 0
total_bytes_compact: 16
total_strided_bytes: 16

Compacting Nodes¶

Nodes can be compacted to transform sparse data.

float64 vals[] = { 100.0,-100.0,
                   200.0,-200.0,
                   300.0,-300.0,
                   400.0,-400.0,
                   500.0,-500.0};

// stride though the data with two different views. 
Generator g1("{dtype: float64, length: 5, stride: 16}",
             "conduit_json",
             vals);
Generator g2("{dtype: float64, length: 5, stride: 16, offset:8}",
             "conduit_json",
              vals);

Node n1;
g1.walk_external(n1);
n1.print();

Node n2;
g2.walk_external(n2);
n2.print();

// look at the memory space info for our two views
Node ninfo;
n1.info(ninfo);
ninfo.print();

n2.info(ninfo);
ninfo.print();

// compact data from n1 to a new node
Node n1c;
n1.compact_to(n1c);

// look at the resulting compact data
n1c.print();
n1c.schema().print();
n1c.info(ninfo);
ninfo.print();

// compact data from n2 to a new node
Node n2c;
n2.compact_to(n2c);

// look at the resulting compact data
n2c.print();
n2c.info(ninfo);
ninfo.print();

[100.0, 200.0, 300.0, 400.0, 500.0]
[-100.0, -200.0, -300.0, -400.0, -500.0]

mem_spaces: 
  0x7ffee7d56100: 
    path: ""
    type: "external"
total_bytes_allocated: 0
total_bytes_mmaped: 0
total_bytes_compact: 40
total_strided_bytes: 72


mem_spaces: 
  0x7ffee7d56100: 
    path: ""
    type: "external"
total_bytes_allocated: 0
total_bytes_mmaped: 0
total_bytes_compact: 40
total_strided_bytes: 72

[100.0, 200.0, 300.0, 400.0, 500.0]
{"dtype":"float64","number_of_elements": 5,"offset": 0,"stride": 8,"element_bytes": 8,"endianness": "little"}

mem_spaces: 
  0x7fefae904610: 
    path: ""
    type: "allocated"
    bytes: 40
    allocator_id: 0
total_bytes_allocated: 40
total_bytes_mmaped: 0
total_bytes_compact: 40
total_strided_bytes: 40

[-100.0, -200.0, -300.0, -400.0, -500.0]

mem_spaces: 
  0x7fefae904640: 
    path: ""
    type: "allocated"
    bytes: 40
    allocator_id: 0
total_bytes_allocated: 40
total_bytes_mmaped: 0
total_bytes_compact: 40
total_strided_bytes: 40