Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Arrow, pandas and polars

GraphDb.to_arrow_tables() returns the whole graph as two Apache Arrow tables, so a data tool can read it without a Cypher round trip.

from mushroomdb import GraphDb

db = GraphDb.open("./store")
tables = db.to_arrow_tables()      # {"nodes": pyarrow.Table, "edges": pyarrow.Table}

nodes_df = tables["nodes"].to_pandas()             # pandas

import polars as pl
edges_df = pl.from_arrow(tables["edges"])          # polars

pyarrow is optional. Install it with pip install pyarrow; without it the call raises ImportError naming that command. pandas and polars are yours to install as well.

Columns

The columns are fixed, so a script written against one store reads any other.

TableColumnTypeMeaning
nodeskeystringThe node key. Rows are sorted by it.
nodeslabelstringThe node’s label.
nodespropertiesstringA JSON object of the node’s properties, keys sorted, always valid JSON. Each value is written as the HTTP API writes it; see below.
edgessourcestringKey of the node the edge leaves.
edgestypestringThe edge type.
edgestargetstringKey of the node the edge enters.
edgesderivedbooltrue for an edge a rule derived, false for one you wrote.
edgesrulestringThe deriving rule’s name; null for a manual edge.
edgesweightfloat64The rule’s weight_prop value when it is numeric; otherwise null.
edgespropertiesstringA JSON object of all edge properties, including stored rule weights and insert count, with the same sorted keys and value forms as node properties. {} when none are stored.

Edges are sorted by (type, source, target). An empty store gives empty tables with the same columns.

Each table has one JSON properties column because different nodes and edges carry different properties. To read them into columns, use pandas.json_normalize(nodes_df["properties"].map(json.loads)) (or edges_df for edges), or polars str.json_decode(dtype) with a pl.Struct naming the properties you want. The weight metadata column describes the deriving rule; a manual property named weight appears in properties while the metadata stays null.

Property value forms

The forms match what the HTTP API returns for the same value, so the two surfaces agree.

Engine valueJSON in properties
IntegerA JSON number, exact up to 64 bits. Parse it as a 64-bit integer: a double-based parser loses digits above 2^53.
FloatA JSON number. NaN, Infinity and -Infinity are null, never NaN.
Boolean, stringtrue / false, a JSON string.
List, mapA JSON array, a JSON object, with these same forms inside.
Null inside a list or mapnull
Date{"$date": "2024-01-15"}
Local date-time{"$ldt": "2024-01-15T12:34:56.789012"}
Date-time with offset{"$dt": "2024-01-15T12:34:56.789012+05:30"}
Duration{"$dur": "P1Y2M3DT4S"}, ISO 8601
Binary{"$binary": "AP8="}, base64

A date outside Python’s datetime range (year above 9999) still exports, as its text form.

Namespaces and scope

The namespace a node lives in is a property: it appears in properties as ns. An unscoped handle exports every namespace, the same as an unscoped db.query, and ignores roles and masks. A handle from scoped() refuses the call.

Limits

  • It materialises every node and edge in memory at once. Use it on a graph that fits in memory, or query a slice with db.query instead.
  • It is a read of the whole store, so a handle from scoped() refuses it with ValueError; call it on the handle scoped() was called on.
  • It reads the live store and takes no at= argument. For a past commit, use query_at.

For a complete script that builds a graph and retrieves from it, see examples/graph-retrieval/ in the repository.