Rockfish Entity Data Generator

The rockfish.actions.ent module generates synthetic data tables directly from a declarative schema. You describe the entities (tables), their columns, and how they relate; Rockfish generates realistic values, relationships, and temporal patterns. This is useful for populating test and development environments, building demos, and producing shareable datasets that carry no real customer data.

Overview

The GenerateFromDataSchema action builds one PyArrow table per entity from a single DataSchema and (optionally) uploads each as a labeled Rockfish dataset. A schema can express:

Multiple entities (tables) linked by foreign keys, including composite keys and hierarchical parent/child fan-out (one parent row expanding into many child rows).
Independent columns: IDs and globally-unique keys, categorical values, statistical distributions, and realistic named entities (names, emails, addresses, and more).
Stateful columns: time-varying measurements (timeseries) and behavioral sequences (state machines).
Derived columns: foreign keys, arithmetic, value mapping, cross-entity aggregations, timestamp reformatting, and string templates.
Temporal data over a configurable global time window, emitted as timezone-aware UTC timestamps.
Reproducible output: a single global seed makes an entire run deterministic.

Quick Start

A minimal schema that generates a metadata-only user table:

import asyncio
import rockfish as rf
import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    IDParams,
    NormalDistParams,
)


async def main():
    # Define the schema using typed Python objects
    schema = DataSchema(
        entities=[
            Entity(
                name="users",
                cardinality=50,
                columns=[
                    Column(
                        name="user_id",
                        data_type="string",
                        column_type=ColumnType.INDEPENDENT,
                        column_category_type=ColumnCategoryType.METADATA,
                        domain=Domain(
                            type=DomainType.ID,
                            params=IDParams(template_str="USER_{id}")
                        )
                    ),
                    Column(
                        name="age",
                        data_type="int64",
                        column_type=ColumnType.INDEPENDENT,
                        column_category_type=ColumnCategoryType.METADATA,
                        domain=Domain(
                            type=DomainType.NORMAL_DIST,
                            params=NormalDistParams(mean=35.0, std=10.0)
                        )
                    )
                ]
            )
        ],
        seed=42,  # optional: makes the run reproducible
    )

    action = ra.GenerateFromDataSchema(
        schema=schema,
        entity_labels={"users": {"use_for": "testing"}}
    )

    builder = rf.WorkflowBuilder()
    builder.add(action)

    async with rf.Connection.from_env() as conn:
        workflow = await builder.start(conn)
        print(f"Workflow ID: {workflow.id()}")

        remote_dataset = await workflow.datasets().nth(0)
        dataset = await remote_dataset.to_local(conn)

        # Save to file
        dataset.to_pandas().to_csv(f"{dataset.name()}.csv", index=False)


asyncio.run(main())

Building a schema

`rockfish.actions.ent.GenerateFromDataSchemaConfig`

Config class for the GenerateFromDataSchema action.

from rockfish.actions.ent import (
    DataSchema, Entity, Column,
    GenerateFromDataSchemaConfig
)

schema = DataSchema(entities=[...])
config = GenerateFromDataSchemaConfig(
    schema=schema,
    entity_labels={"users": {"use_for": "testing"}}
)

Attributes:

Name	Type	Description
`schema`	`DataSchema`	DataSchema configuration defining entities and relationships. Use the rockfish.actions.ent module to construct schema objects programmatically, or provide a dict that will be structured into a DataSchema.
`entity_labels`	`dict[str, LabelDict]`	Optional mapping of entity names to a Rockfish LabelDict. Labels are applied to the generated datasets for organization. Example: {"users": {"use_for": "testing"}, "transactions": {"type": "fraud"}}
`dataset_name_prefix`	`str`	Prefix for dataset names. Default: `""` (no prefix)
`upload_datasets`	`bool`	If True, upload each entity as a dataset. If False, yield tables for downstream actions. Default: `True`.

`rockfish.actions.ent.DataSchema`

Root data schema specification.

The top-level specification defining all entities, their relationships, and a global timestamp configuration (optional for metadata-only entities).

DataSchema configurations must follow specific rules:

Must have at least one entity, entities must have unique names
If any entity has a timestamp, global_timestamp must be defined
Entity relationships must reference valid entities that exist in the schema
Relationship join_columns must reference existing columns in both parent_entity and child_entity
FOREIGN_KEY columns can only appear in child_entity, not parent_entity, unless the parent is itself a child in another relationship (a parent->child->grandchild chain)
ONE_TO_ONE relationships require child_entity cardinality <= parent_entity cardinality
All foreign_key columns must be declared in entity_relationships
A count-driven child (child_count_column) is only valid for ONE_TO_MANY, and a child may be count-driven by at most one relationship; GROUP_ORDINAL columns are only valid on a count-driven child

Complete data schema with entities and relationships

DataSchema(
    entities=[
        Entity(name="users", cardinality=50, columns=[...]),
        Entity(name="sessions", cardinality=200, columns=[...],
               timestamp=Timestamp(column_name="timestamp"))
    ],
    entity_relationships=[
        EntityRelationship(
            parent_entity="users",
            child_entity="sessions",
            relationship_type=EntityRelationshipType.ONE_TO_MANY,
            join_columns={"user_id": "user_id"}
        )
    ],
    global_timestamp=GlobalTimestamp(
        t_start="2025-01-01T00:00:00Z",
        t_end="2025-01-01T01:00:00Z",
        time_interval="1min"
    ),
    seed=42
)

Attributes:

Name	Type	Description
`entities`	`list[Entity]`	List of entity specifications
`entity_relationships`	`list[EntityRelationship]`	List of relationships between entities. Default: `[]` (empty list)
`global_timestamp`	`Optional[GlobalTimestamp]`	Global timestamp configuration for entities with measurements. Default: `None`
`seed`	`Optional[int]`	Global random seed. When set, every random stream in the run derives from it and the same schema produces identical output. When `None`, the server draws a fresh run seed, logs it, and attaches it to each uploaded dataset as an `ent_seed` label, so any run can be reproduced by setting this field to that value. Component-level seeds (e.g. `CategoricalParams.seed`, `GlobalTimestamp.seed`) override the derived stream for that component. Requires a server running 0.73.0 or later; older servers ignore this field. Default: `None`
`scale_factor`	`float`	global volume multiplier applied to the `cardinality` of every entity whose `scale_with_factor` is True (fact/event entities), leaving reference/dimension entities fixed. Lets one schema generate a small sample or a large run without editing per-entity counts. Must be a finite positive number. Default: `1.0`

Key validation rules:

At least one entity must be defined.
Entity names must be unique.
If any entity has a timestamp, global_timestamp must be provided.
All relationship references must point to valid entities.

Reproducibility

Set DataSchema.seed to make an entire run deterministic: the same schema and seed produce identical output every time. When seed is left unset, the server draws a fresh seed, logs it, and attaches it to each uploaded dataset as an ent_seed label, so any run can be reproduced later by setting seed to that value.

Per-component seeds (for example CategoricalParams.seed or NormalDistParams.seed) override the global stream for that one column, which is useful when you want most of a schema to vary run-to-run but a few columns to stay fixed.

Scaling the volume

Set DataSchema.scale_factor to grow or shrink an entire run from one schema without editing per-entity counts: it multiplies the cardinality of every entity whose Entity.scale_with_factor is True. Leave scale_with_factor=True (the default) on fact/event entities that should grow with the run, and set it to False on reference/dimension entities (a fixed catalog of products, merchants, or devices) so they stay the same size as the run scales. Count-driven children ignore cardinality entirely and scale through their parent's row count. scale_factor must be a finite positive number (for example 10.0 for a ten-times-larger run).

`rockfish.actions.ent.Entity`

Entity specification.

Defines a complete entity (table) with its cardinality, columns, and optional timestamp configuration for time-series data.

Entity configurations must follow specific rules:

Entities with measurements should have a timestamp, and vice versa
Derived columns must reference dependent columns within the same entity, except for the SAMPLE_FROM_COLUMN derivation function type
A metadata column cannot depend on a measurement column (metadata is generated before timestamp expansion); a measurement column may depend on metadata

Metadata-only entity (no timestamps)

Entity(
    name="users",
    cardinality=50,
    columns=[
        Column(
            name="user_id",
            data_type="string",
            column_type=ColumnType.INDEPENDENT,
            column_category_type=ColumnCategoryType.METADATA,
            domain=Domain(type=DomainType.ID, params=IDParams(template_str="USER_{id}"))
        )
    ]
)

Entity with measurements and a primary timestamp

Entity(
    name="sessions",
    cardinality=200,
    timestamp=Timestamp(column_name="timestamp"),
    columns=[
        Column(
            name="session_id",
            data_type="string",
            column_type=ColumnType.INDEPENDENT,
            column_category_type=ColumnCategoryType.METADATA,
            domain=Domain(type=DomainType.ID, params=IDParams(template_str="SESSION_{id}"))
        ),
        Column(
            name="response_time",
            data_type="float64",
            column_type=ColumnType.STATEFUL,
            column_category_type=ColumnCategoryType.MEASUREMENT,
            domain=Domain(type=DomainType.TIMESERIES, params=TimeseriesParams(...))
        )
    ]
)

Attributes:

Name	Type	Description
`name`	`str`	Entity name (e.g., "users", "sessions", "transactions")
`cardinality`	`int`	Number of rows/instances to generate
`columns`	`list[Column]`	List of column specifications
`timestamp`	`Optional[Timestamp]`	Timestamp configuration for entities with measurements. Default: `None`
`scale_with_factor`	`bool`	whether `DataSchema.scale_factor` multiplies this entity's `cardinality`. Fact/event entities scale with the run (more events); set it `False` on reference/dimension entities so they stay a fixed catalog as the run grows. Count-driven children ignore `cardinality` and scale through their parent's row count regardless. Default: `True`

Key validation rules:

Cardinality must be positive.
At least one column must be defined.
Column names must be unique within an entity.
An entity has a timestamp if and only if it has at least one measurement column.

Entity relationships

`rockfish.actions.ent.EntityRelationship`

Specification for relationships between entities.

Defines how two entities are related through foreign key columns.

Many sessions belong to one user

EntityRelationship(
    parent_entity="users",
    child_entity="sessions",
    relationship_type=EntityRelationshipType.ONE_TO_MANY,
    join_columns={"user_id": "user_id"}
)

Composite foreign key

EntityRelationship(
    parent_entity="transport_interfaces",
    child_entity="cell_sites",
    relationship_type=EntityRelationshipType.ONE_TO_MANY,
    join_columns={
        "device_id": "transport_device_id",
        "interface_id": "transport_interface_id"
    }
)

Attributes:

Name	Type	Description
`parent_entity`	`str`	Name of the parent entity (contains the primary key)
`child_entity`	`str`	Name of the child entity (contains the foreign key)
`relationship_type`	`EntityRelationshipType`	Type of relationship (one_to_one, one_to_many)
`join_columns`	`dict[str, str]`	Mapping from parent entity PK column(s) to child entity FK column(s). Keys are column names in parent_entity (PK), values are column names in child_entity (FK). Default: `{}` (empty dict)
`child_count_column`	`Optional[str]`	Optional name of a column in `parent_entity` whose per-row integer value is the number of child rows to emit for that parent row. When set, the child's row count is driven by the parent (one parent row with value `k` produces exactly `k` child rows), the child inherits the parent's `join_columns` values on each of those rows, and the child entity's declared `cardinality` is ignored. Only valid for ONE_TO_MANY. Used to explode a count into rows (orders to line-items, windows to events); a parent counter equal to this column then matches the child row count by construction. Default: `None`
`inherit_columns`	`dict[str, str]`	optional map of `parent_column -> child_column` whose values the child copies from the SAME parent row it joins to, rather than sampling them independently. Use it for denormalized keys/attributes a child shares with its parent (a line-item carrying its order's customer key), so those columns agree with the joined parent by construction. The inherited value may be any parent attribute (a key or a plain field such as a region), but the child column that receives it must be declared `foreign_key` (it holds a value copied from the parent rather than one drawn from its own domain).
`self_reference`	`bool`	when True the relationship is from an entity to itself (a hierarchy or chain: an employee's manager, a comment's parent comment). `parent_entity` must equal `child_entity`, the single `join_columns` pair maps the entity's key column to the self-referencing foreign-key column, and each row points at a strictly-earlier row (null for roots), so the reference is acyclic by construction.
`partition_by`	`Optional[str]`	with `self_reference`, restrict a row's reference to strictly-earlier rows sharing this metadata column's value (e.g. a reply references an earlier comment WITHIN THE SAME thread).
`root_fraction`	`float`	with `self_reference`, the fraction of non-forced rows made roots (null reference) in addition to the first row of each partition.
`weight_column`	`Optional[str]`	optional column in `parent_entity` giving each parent row's relative likelihood of being chosen by a child. When set, children are assigned to parents in proportion to this weight rather than uniformly, so a heavily weighted parent becomes a high-volume "whale" and activity is skewed (Pareto- like) as in real data. Only for a sampled foreign key (no `child_count_column`) on a one-to-many relationship; the column must be numeric.
`affinity_column`	`Optional[str]`	optional column in `parent_entity` used for affinity ("homophily") sampling: with probability `affinity_prob` a child is matched to a parent whose `affinity_column` equals the child's own `affinity_local_column` value (e.g. a transaction prefers a merchant in the cardholder's country), and otherwise to any parent. Within the chosen pool, `weight_column` still applies. Requires `affinity_local_column`; only for a sampled foreign key on a one-to-many relationship.
`affinity_local_column`	`Optional[str]`	the metadata column on the child holding the value matched against the parent's `affinity_column`. Must already be populated when the foreign key is sampled (e.g. an inherited or looked-up attribute), and cannot be a column this same relationship fills.
`affinity_prob`	`float`	probability in [0, 1] that a child is matched to a same-value parent; the remainder are drawn from all parents. 0 disables affinity.

Composite foreign keys:

When join_columns contains multiple column pairs, Rockfish samples matching tuples from the parent entity and preserves referential integrity across all of them. All foreign-key columns must be declared as column_type=ColumnType.FOREIGN_KEY with no domain or derivation; the relationship supplies their values.

Count-driven fan-out:

Set child_count_column to explode each parent row into a variable number of child rows. The named column lives on the parent and holds a per-row integer count; a parent row whose count is k produces exactly k child rows, each inheriting the parent's join_columns values. When a relationship is count-driven, the child entity's declared cardinality is ignored and its row count comes entirely from the parent. This is the natural way to model orders and their line items, sessions and their events, or devices and their interfaces. Count-driven fan-out is only valid for ONE_TO_MANY relationships, and it pairs with the GROUP_ORDINAL domain (a per-parent index on the child) and the AGGREGATE_FROM_CHILD derivation (rolling child rows back up to the parent).

Inherited columns:

Set inherit_columns (a parent_column -> child_column map) to copy denormalized values from the SAME parent row a child joins to, instead of sampling them independently. This keeps a shared key or attribute (for example a line item carrying its order's customer key, or a transaction carrying its merchant's risk rating) in agreement with the parent by construction. The child column that receives the value must be declared column_type=ColumnType.FOREIGN_KEY.

Self-references (hierarchies and chains):

Set self_reference=True for a relationship from an entity to itself, so each row points at the key of a strictly-earlier row (an employee's manager, a comment's parent comment). parent_entity must equal child_entity, and the single join_columns pair maps the entity's key column to its self-referencing foreign-key column; the result is acyclic by construction. Use partition_by to confine references within a group (a reply references an earlier comment in the same thread), and root_fraction to leave a fraction of rows as roots (null reference).

Weighted parent sampling (whales):

Set weight_column to a numeric column on the parent so children are assigned to parents in proportion to that weight rather than uniformly. A heavily weighted parent becomes a high-volume "whale" and activity is skewed (Pareto-like), as in real data. Only valid for a sampled foreign key (no child_count_column) on a ONE_TO_MANY relationship.

Affinity (co-location) sampling:

Set affinity_column (a parent column), affinity_local_column (a matching metadata column on the child), and affinity_prob so that, with that probability, a child is matched to a parent sharing its value (for example a transaction preferring a merchant in the cardholder's own country), and otherwise to any parent. Weighting still applies within the chosen pool. This models homophily / mostly-local relationships.

`rockfish.actions.ent.EntityRelationshipType`

Source code in src/rockfish/actions/ent/generate.py

class EntityRelationshipType(str, Enum):
    ONE_TO_ONE = "one_to_one"
    ONE_TO_MANY = "one_to_many"

Relationship semantics (from the parent's perspective):

ONE_TO_ONE: Each instance in parent_entity relates to exactly one unique instance in child_entity.
ONE_TO_MANY: Each instance in parent_entity can be referenced by multiple instances in child_entity.

Timestamps

`rockfish.actions.ent.Timestamp`

Timestamp specification for entities with measurements.

Specifies that an entity should have timestamps, and what the timestamp column should be called. The actual timestamp range and interval are defined in the global_timestamp.

Timestamp with default data type

Timestamp(column_name="timestamp")

Timestamp with custom column name

Timestamp(column_name="event_time", data_type="timestamp")

Attributes:

Name	Type	Description
`column_name`	`str`	Name of the timestamp column (e.g., "timestamp", "event_time")
`data_type`	`str`	Data type for the timestamp column. Default: `"timestamp"`

`rockfish.actions.ent.GlobalTimestamp`

Global timestamp specification for entities.

Defines the time range and interval for all entities with timestamps.

The window [t_start, t_end] is discretized into a grid of ticks spaced time_interval apart. For entities with state machine columns, each session is placed at a random start tick within the window and its events occupy consecutive ticks from there, so time_interval is the spacing between consecutive events within a session and sessions distribute across the full window; other measurement columns are sampled at the session's ticks. Entities whose only stateful columns are timeseries expand densely: every session has one row at every tick.

Timezone contract: t_start and t_end accept ISO 8601 with a Z suffix, an explicit UTC offset, or no timezone at all. Naive bounds are interpreted as UTC; offset-bearing bounds are converted to UTC. Every generated timestamp is timezone-aware UTC, rendered in ISO 8601 with a +00:00 offset, regardless of how the bounds were written (server 0.77.0 or later; older servers mirror the timezone style of t_start).

Define global timestamp with interval

GlobalTimestamp(
    t_start="2025-01-01T00:00:00Z",
    t_end="2025-01-01T01:00:00Z",
    time_interval="1min"
)

Reproducible session placement with a seed

GlobalTimestamp(
    t_start="2025-01-01T00:00:00Z",
    t_end="2025-01-15T00:00:00Z",
    time_interval="1hour",
    seed=42
)

Attributes:

Name	Type	Description
`t_start`	`str`	Start timestamp in ISO 8601 format (e.g., "2025-01-01T00:00:00Z")
`t_end`	`str`	End timestamp in ISO 8601 format (e.g., "2025-01-01T23:59:59Z")
`time_interval`	`str`	Time interval between measurements (e.g., "1min", "15min", "1hour")
`seed`	`Optional[int]`	Random seed for session start placement and state machine transitions. Runs with the same seed produce identical output; when `None`, the timestamp path derives its seed from `DataSchema.seed` (or the per-run seed drawn when that is also unset). Requires a server running 0.73.0 or later; older servers ignore this field. Default: `None`

Supported time interval formats:

"1min", "5min", "15min", and so on
"1hour", "2hour", and so on
"1day", "7day", and so on
"1month", "3month", and so on

Timezone contract: t_start and t_end accept ISO 8601 with a Z suffix, an explicit UTC offset, or no timezone at all. Naive bounds are interpreted as UTC and offset-bearing bounds are converted to UTC. Every generated timestamp is emitted as timezone-aware UTC in ISO 8601 form.

Columns

`rockfish.actions.ent.Column`

Column specification within an entity.

Defines a column's type, data type, category, and how its values are generated (via domain for independent/stateful columns or derivation for derived columns).

Column configurations must follow specific rules based on column type and category:

Independent columns require a non-temporal domain and must be metadata
Stateful columns require a temporal domain (STATE_MACHINE or TIMESERIES) and must be measurement
Derived columns require a derivation and can be metadata or measurement
Foreign key columns must be metadata and cannot have domain or derivation

Independent column

Column(
    name="user_id",
    data_type="string",
    column_type=ColumnType.INDEPENDENT,
    column_category_type=ColumnCategoryType.METADATA,
    domain=Domain(
        type=DomainType.ID,
        params=IDParams(template_str="USER_{id}")
    )
)

Derived column

Column(
    name="total",
    data_type="float64",
    column_type=ColumnType.DERIVED,
    column_category_type=ColumnCategoryType.MEASUREMENT,
    derivation=Derivation(
        function_type=DerivationFunctionType.SUM,
        dependent_columns=["amount1", "amount2"],
        params=SumParams()
    )
)

Attributes:

Name	Type	Description
`name`	`str`	Column name
`data_type`	`str`	String alias for a pyarrow data type (e.g., "string", "int64", "float64", "timestamp"). Data from the specified domain will be cast to this type on a best-effort basis.
`column_type`	`ColumnType`	Type of column (independent, stateful, derived, foreign_key)
`column_category_type`	`ColumnCategoryType`	Data model category type for column (metadata or measurement)
`domain`	`Optional[Domain]`	Domain specification (for independent/stateful columns only)
`derivation`	`Optional[Derivation]`	Derivation specification (for derived columns only)

The column category can be metadata or measurement (see supported data models for examples).

A column's data_type is an Arrow type alias such as "string", "int64", "float64", or "timestamp[us]". For exact monetary values, use a parameterized decimal, e.g. "decimal128(18, 2)", so amounts stay precise to the cent rather than carrying floating-point error.

`rockfish.actions.ent.ColumnCategoryType`

Source code in src/rockfish/actions/ent/generate.py

class ColumnCategoryType(str, Enum):
    METADATA = "metadata"
    MEASUREMENT = "measurement"

`rockfish.actions.ent.ColumnType`

Source code in src/rockfish/actions/ent/generate.py

class ColumnType(str, Enum):
    INDEPENDENT = "independent"
    STATEFUL = "stateful"
    DERIVED = "derived"
    FOREIGN_KEY = "foreign_key"

Independent: Generated on its own from a domain. Cannot use temporal domains and must be metadata category.
Stateful: Temporal columns using timeseries or state machines. Must be measurement category.
Derived: Computed from other columns (or from a child entity) via a derivation function. Can be metadata or measurement.
Foreign key: References another entity. Must be metadata category and carries no domain or derivation.

Domains

A domain defines how an independent or stateful column's values are produced. Pick the domain that matches the shape of the data you want:

Category	Domain types	Use for
Identifiers and keys	`ID`, `SEQUENTIAL_INT`, `UNIQUE`, `GROUP_ORDINAL`	Primary keys, templated IDs, globally-unique MAC/IP/keys, per-parent child indices
Categorical and realistic values	`CATEGORICAL`, `NAMED_ENTITY_PROVIDER`	Enumerated choices, and realistic names, emails, addresses, and similar
Statistical distributions	`UNIFORM_DIST`, `NORMAL_DIST`, `LOGNORMAL_DIST`, `EXPONENTIAL_DIST`	Numeric attributes with a known shape (symmetric, right-skewed, waiting times)
Structured and composite	`MIXTURE`, `SEQUENCE`	Multimodal columns; variable-length token sequences
Temporal (stateful columns only)	`TIMESERIES`, `STATE_MACHINE`	Time-varying measurements and behavioral state progressions

GROUP_ORDINAL is only valid on a count-driven child (see count-driven fan-out). TIMESERIES and STATE_MACHINE are the only domains permitted on stateful (measurement) columns; every other domain is used on independent (metadata) columns.

`rockfish.actions.ent.DomainType`

Source code in src/rockfish/actions/ent/generate.py

class DomainType(str, Enum):
    ID = "id"
    SEQUENTIAL_INT = "sequential_int"
    CATEGORICAL = "categorical"
    UNIFORM_DIST = "uniform_dist"
    NORMAL_DIST = "normal_dist"
    EXPONENTIAL_DIST = "exponential_dist"
    LOGNORMAL_DIST = "lognormal_dist"
    STATE_MACHINE = "state_machine"
    TIMESERIES = "timeseries"
    NAMED_ENTITY_PROVIDER = "named_entity_provider"
    UNIQUE = "unique"
    GROUP_ORDINAL = "group_ordinal"
    SEQUENCE = "sequence"
    MIXTURE = "mixture"

`rockfish.actions.ent.Domain`

Domain specification for independent and stateful columns.

Categorical domain

Domain(
    type=DomainType.CATEGORICAL,
    params=CategoricalParams(
        values=["alice", "bob", "charlie"],
        with_replacement=False
    )
)

Attributes:

Name	Type	Description
`type`	`DomainType`	The type of domain/generator to use
`params`	`Union[IDParams, SequentialIntParams, CategoricalParams, UniformDistParams, NormalDistParams, ExponentialDistParams, LogNormalDistParams, TimeseriesParams, StateMachineParams, NamedEntityProviderParams, UniqueParams, GroupOrdinalParams, SequenceParams, MixtureParams]`	Typed parameters specific to the domain type

Domain parameters

Each domain type takes a matching parameters object.

`rockfish.actions.ent.IDParams`

Parameters for ID domain generation. Generates unique ID strings using a template with {id} placeholder.

Create ID parameters

IDParams(template_str="USER_{id}")

Attributes:

Name	Type	Description
`template_str`	`str`	Format string with {id} placeholder (e.g., "USER_{id}"). Default: `"id_{id}"`

`rockfish.actions.ent.SequentialIntParams`

Parameters for sequential integer ID generation.

Sequential integers starting from 1

SequentialIntParams(start=1)

Sequential integers starting from 100

SequentialIntParams(start=100)

Attributes:

Name	Type	Description
`start`	`int`	Starting value for the sequence. Default: `1`

`rockfish.actions.ent.UniqueParams`

Parameters for the unique domain: globally-unique values allocated from a contiguous index range.

Uniqueness within the column is guaranteed because each value derives from a distinct index. format selects the rendering: a {index} template, a MAC address (first octet mac_prefix, remaining bits from the index), or an IPv4 address (ipv4_base plus the index). Useful for keys, MACs, and IPs in device/network data.

Unique device keys

UniqueParams(format="template", template_str="DEV-{index:06d}")

Locally-administered MAC addresses

UniqueParams(format="mac", mac_prefix="02")

Attributes:

Name	Type	Description
`format`	`Literal['template', 'mac', 'ipv4']`	One of `"template"`, `"mac"`, or `"ipv4"`. Default: `"template"`
`template_str`	`str`	Index template for `format="template"` (must contain an `{index}` field). Default: `"id-{index}"`
`start`	`int`	First index, to avoid colliding with an existing range. Default: `0`
`mac_prefix`	`str`	First MAC octet (two hex digits) for `format="mac"`; use a locally-administered value (`02`/`06`/`0A`/`0E`). Default: `"02"`
`ipv4_base`	`str`	Base address for `format="ipv4"`. Default: `"10.0.0.0"`

`rockfish.actions.ent.GroupOrdinalParams`

Parameters for the group-ordinal domain: a count-driven child's 0-based index within its parent group.

When a parent row fans out into K child rows, those children are numbered start, start + 1, ..., start + K - 1. Use it to give each child a distinct ordinal under its parent (e.g. a per-device radio index, or a line number within an order). Only valid on a count-driven child entity (an entity whose row count is driven by a parent's child_count_column).

Per-parent ordinal starting at 0

GroupOrdinalParams(start=0)

Attributes:

Name	Type	Description
`start`	`int`	Value of the first child in each group. Default: `0`

`rockfish.actions.ent.CategoricalParams`

Parameters for categorical value sampling.

Sample from categorical values without replacement

CategoricalParams(
    values=["alice", "bob", "charlie"],
    with_replacement=False
)

Attributes:

Name	Type	Description
`values`	`list[Any]`	List of categorical values to sample from
`weights`	`Optional[list[float]]`	Probability weights for each value (will be normalized). Default: `None`
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`
`with_replacement`	`bool`	If True, allow repeated values; if False, sample without replacement. Default: `True`

`rockfish.actions.ent.NamedEntityProviderParams`

Parameters for the named-entity provider domain. Generates a pool of realistic values using Mimesis (primary) or Faker (fallback), then samples from that pool.

See NamedEntityProvider for common provider constants. Any Mimesis or Faker provider path may be passed as a string.

First names with locale and reproducible seed

NamedEntityProviderParams(
    provider=NamedEntityProvider.PERSON_FIRST_NAME,
    locale="en_us",
    seed=42,
)

Unique emails for a large user pool

NamedEntityProviderParams(
    provider=NamedEntityProvider.PERSON_EMAIL,
    unique_values=5000,
    with_replacement=False,
    seed=1,
)

UUIDs as primary keys

NamedEntityProviderParams(
    provider=NamedEntityProvider.CRYPTOGRAPHIC_UUID,
    unique_values=10000,
    with_replacement=False,
    seed=99,
)

Attributes:

Name	Type	Description
`provider`	`str`	Provider path. Mimesis providers use a dot-path (`"person.first_name"`); Faker-only providers use a bare name (`"ssn"`, `"iban"`). Case-insensitive on the server side.
`locale`	`str`	Locale for generated values. Default: `"en"`
`unique_values`	`int`	Maximum pool size; the actual pool may be smaller for low-cardinality providers. Default: `100`
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`
`with_replacement`	`bool`	If True, allow repeated values; if False, sample without replacement (auto-forced True when rows exceed pool size). Default: `True`

`rockfish.actions.ent.NamedEntityProvider`

Common provider strings for NamedEntityProviderParams.

Pass any Mimesis or Faker provider path as a string; these constants are provided for autocomplete and discoverability of the common ones. The provider field accepts any string, so providers not enumerated here (including those added in future Mimesis releases) work without an SDK update.

Equivalent ways to specify a provider

NamedEntityProviderParams(provider=NamedEntityProvider.PERSON_FIRST_NAME)
NamedEntityProviderParams(provider="person.first_name")

`rockfish.actions.ent.UniformDistParams`

Parameters for uniform distribution generation.

Uniform distribution between 0 and 10

UniformDistParams(lower=0.0, upper=10.0)

Attributes:

Name	Type	Description
`lower`	`float`	Lower bound (inclusive)
`upper`	`float`	Upper bound (exclusive)
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

`rockfish.actions.ent.NormalDistParams`

Parameters for normal (Gaussian) distribution generation.

Normal distribution with mean 100 and std 15

NormalDistParams(mean=100.0, std=15.0)

Attributes:

Name	Type	Description
`mean`	`float`	Mean (center) of the distribution
`std`	`float`	Standard deviation (spread) of the distribution
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

`rockfish.actions.ent.LogNormalDistParams`

Parameters for lognormal distribution generation: exp(Normal(mu, sigma)).

mu and sigma are the mean and standard deviation of the underlying normal in natural-log space (not the mean/std of the lognormal itself). The standard model for strictly-positive, right-skewed quantities that span orders of magnitude: latencies, durations, sizes, incomes. A narrow sigma approximates a soft spike; a wide one a heavy right tail.

Lognormal latency in log-space (median ~exp(3.4))

LogNormalDistParams(mu=3.4, sigma=0.8)

Attributes:

Name	Type	Description
`mu`	`float`	Mean of the underlying normal in natural-log space
`sigma`	`float`	Standard deviation of the underlying normal in natural-log space (must be positive)
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

`rockfish.actions.ent.ExponentialDistParams`

Parameters for exponential distribution generation. Often used for modeling time between events or waiting times.

Exponential distribution with scale 2.0

ExponentialDistParams(scale=2.0)

Attributes:

Name	Type	Description
`scale`	`float`	Scale parameter (1/lambda), controls the mean of the distribution
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

`rockfish.actions.ent.MixtureParams`

Parameters for the mixture domain: a weighted mixture of sub-distributions (a mixture model).

For each row a component is chosen by its (normalized) weight and one value is drawn from that component's domain. Models a multimodal column (e.g. a latency that is part instant rejects, part a spike, part a long tail) as a blend of simpler domains. Components may be any sampling domain except whole-column allocators (UNIQUE / GROUP_ORDINAL), and may themselves be mixtures.

A point mass at 0 blended with a lognormal tail

MixtureParams(
    components=[
        {"weight": 0.3, "domain": Domain(type=DomainType.CATEGORICAL,
                                         params=CategoricalParams(values=[0.0]))},
        {"weight": 0.7, "domain": Domain(type=DomainType.LOGNORMAL_DIST,
                                         params=LogNormalDistParams(mu=2.0, sigma=0.5))},
    ]
)

Attributes:

Name	Type	Description
`components`	`list`	List of `{"weight": float, "domain": Domain}`. Weights need not sum to 1; they are normalized.
`seed`	`Optional[int]`	Random seed for the per-row component assignment. Default: `None`

`rockfish.actions.ent.SequenceParams`

Parameters for the sequence domain: a variable-length token sequence rendered as a string.

Each row's value is built by concatenating segments in order; a segment contributes its tokens repeated a per-row count drawn uniformly from [min_repeat, max_repeat]. The assembled token list is encoded to a string ("json" produces a compact JSON array like ["a","b"]; "delimited" joins tokens by separator). Models structured sequences such as event / frame / retry traces: a fixed prefix, a body repeated N times, and a tail.

A prefix, a body repeated 1-3 times, then a tail

SequenceParams(
    segments=[
        {"tokens": ["start"]},
        {"tokens": ["retry"], "min_repeat": 1, "max_repeat": 3},
        {"tokens": ["done"]},
    ],
    encoding="json",
)

Attributes:

Name	Type	Description
`segments`	`list`	Ordered list of dicts, each `{"tokens": [...], "min_repeat": int = 1, "max_repeat": int = min_repeat}`
`encoding`	`Literal['json', 'delimited']`	`"json"` or `"delimited"`. Default: `"json"`
`separator`	`str`	Token separator for `encoding="delimited"`. Default: `","`
`seed`	`Optional[int]`	Random seed for the per-row repeat counts. Default: `None`

`rockfish.actions.ent.TimeseriesParams`

Parameters for timeseries generation with seasonality patterns.

Timeseries with symmetric seasonality

TimeseriesParams(
    base_value=150.0,
    min_value=50.0,
    max_value=300.0,
    seasonality_type="symmetric",
    seasonality_strength=0.3,
    noise_level=0.2
)

Attributes:

Name	Type	Description
`base_value`	`float`	Central value around which the series oscillates
`min_value`	`float`	Minimum value used to clip final values
`max_value`	`float`	Maximum value used to clip final values
`seasonality_type`	`Literal['symmetric', 'peak_offpeak', 'none']`	Type of seasonal pattern ("symmetric", "peak_offpeak", "none"). Default: `"symmetric"`
`peak_start_hour`	`int`	Start hour for peak_offpeak type. Default: `8`
`peak_end_hour`	`int`	End hour for peak_offpeak type. Default: `22`
`seasonality_strength`	`float`	Strength of seasonal pattern (0-1). Default: `0.3`
`noise_level`	`float`	Amount of random noise (0-1). Default: `0.1`
`spike_probability`	`float`	Probability of anomalous spikes (0-1). Default: `0.0`
`spike_magnitude`	`float`	Magnitude of spikes relative to range (0-1). Default: `0.3`
`interval_minutes`	`int`	Time interval between points. Default: `15`
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

Seasonality types:

"symmetric": Smooth sinusoidal pattern throughout the day.
"peak_offpeak": Higher values between peak_start_hour and peak_end_hour, lower outside that window.
"none": No seasonal pattern, only base value plus noise.

`rockfish.actions.ent.StateMachineParams`

State machine definition for generating session-based timeseries data. Models sequential behavior patterns such as user browsing sessions, transaction flows, and system state progressions.

Simple e-commerce browsing state machine

sm = StateMachineParams(
    trigger_column_name="action",
    initial_state="homepage",
    states=["homepage", "search", "product", "cart", "checkout", "exit"],
    terminal_states=["exit"],
    transitions=[
        Transition(trigger="browse", source="homepage", dest="search", probability=0.6),
        Transition(trigger="view_product", source="homepage", dest="product", probability=0.3),
        Transition(trigger="leave", source="homepage", dest="exit", probability=0.1),
        Transition(trigger="add_to_cart", source="product", dest="cart", probability=0.3),
        Transition(trigger="checkout", source="cart", dest="checkout", probability=0.6),
        Transition(trigger="complete", source="checkout", dest="exit", probability=0.8),
    ]
)

State machine with context variables

sm_with_context = StateMachineParams(
    trigger_column_name="event",
    initial_state="pending",
    states=["pending", "processing", "shipped", "delivered"],
    terminal_states=["delivered"],
    transitions=[
        Transition(
            trigger="process",
            source="pending",
            dest="processing",
            probability=0.9,
            conditions=["payment_received"],
            context_updates={"in_fulfillment": True}
        ),
    ],
    context_variables={"payment_received": False, "in_fulfillment": False}
)

Note

All states in terminal_states must be present in states
initial_state must be present in states
Each transition's source and dest must be valid states
Multiple transitions from the same source state will have their probabilities normalized
Implicit columns will be created for trigger_column_name and each context variable key. These names must satisfy the following constraints:
- trigger_column_name must differ from column_name (the stateful column)
- Context variable names must differ from both column_name and trigger_column_name
- All implicit column names must not conflict with any other columns in the entity (including timestamp)

Attributes:

Name	Type	Description
`trigger_column_name`	`str`	Name of the column that will store trigger/action values (e.g., "action", "event", "user_action")
`initial_state`	`str`	The starting state for all sessions/sequences
`states`	`list[str]`	Complete list of all valid states in the state machine
`terminal_states`	`list[str]`	List of states that end the session (no outgoing transitions)
`transitions`	`list[Transition]`	List of `Transition` objects defining all possible state changes
`context_variables`	`dict[str, bool]`	Dictionary of boolean context variables with their initial values. Used for conditional transitions. Default: `{}` (empty dict, no context variables)

`rockfish.actions.ent.Transition`

Represents a single transition in a state machine.

Simple transition from homepage to search

t1 = Transition(
    trigger="browse",
    source="homepage",
    dest="search",
    probability=0.6
)

Transition with conditions and context updates

t2 = Transition(
    trigger="checkout",
    source="cart",
    dest="checkout",
    probability=0.6,
    conditions=["has_items"],
    context_updates={"checkout_started": True}
)

Self-loop transition (staying in same state)

t3 = Transition(
    trigger="refine_search",
    source="search",
    dest="search",
    probability=0.2
)

Attributes:

Name	Type	Description
`trigger`	`str`	The action/event that causes this transition (e.g., "browse", "view_product")
`source`	`str`	The originating state (e.g., "homepage", "search")
`dest`	`str`	The destination state (e.g., "product", "cart")
`probability`	`float`	Probability weight for this transition (0 < p <= 1). When multiple transitions share the same source state, probabilities are normalized to sum to 1.0
`conditions`	`list[str]`	List of context variable names that must be True for this transition to be eligible. Default: `[]` (empty list, no conditions)
`context_updates`	`dict[str, bool]`	Dictionary of context variable updates to apply after this transition executes. Keys are variable names, values are booleans. Default: `{}` (empty dict, no updates) !!! note Probabilities are weights, not exact probabilities. If a source state has transitions with probabilities [0.6, 0.3, 0.1], they will be normalized to [0.6, 0.3, 0.1] since they already sum to 1.0. If they were [2, 1, 1], they would normalize to [0.5, 0.25, 0.25].

Important notes:

Probabilities are weights that are automatically normalized; they do not need to sum to 1.0.
Multiple transitions from the same source state have their probabilities normalized together.
Conditions must reference context variables defined in context_variables.
Context updates can enable or disable transitions dynamically.

Derivations

A derivation computes a column's values from other columns, or aggregates a child entity back onto its parent.

Category	Function types	Use for
Arithmetic	`SUM`, `MULTIPLY`, `ROUND`	Totals and products of numeric columns; rounding a value to a fixed number of decimals (e.g. money to cents)
Referential and conditional sampling	`SAMPLE_FROM_COLUMN`, `SAMPLE_FROM_COLUMN_WHERE`, `CONDITIONAL_SAMPLE`	Foreign keys sampled from another column; a correlated foreign key restricted to rows matching a local value; distributions that vary by category
Mapping and formatting	`MAP_VALUES`, `COPY`, `FORMAT_TIMESTAMP`, `SHIFT_TIMESTAMP`, `STRING_TEMPLATE`, `SUBSTRING`, `LUHN_APPEND`	Value lookups, aliases, timestamp reformatting, shifting a timestamp by a per-row offset, composed strings, a fixed-position substring, and appending a Luhn check digit to an identifier
Cross-row and cross-entity aggregation	`CUMULATIVE`, `AGGREGATE_FROM_CHILD`	Running totals within a session; counts or sums rolled up from a child entity

`rockfish.actions.ent.DerivationFunctionType`

Source code in src/rockfish/actions/ent/generate.py

class DerivationFunctionType(str, Enum):
    SUM = "sum"
    MULTIPLY = "multiply"
    SAMPLE_FROM_COLUMN = "sample_from_column"
    SAMPLE_FROM_COLUMNS = "sample_from_columns"
    MAP_VALUES = "map_values"
    CUMULATIVE = "cumulative"
    CONDITIONAL_SAMPLE = "conditional_sample"
    COPY = "copy"
    AGGREGATE_FROM_CHILD = "aggregate_from_child"
    FORMAT_TIMESTAMP = "format_timestamp"
    STRING_TEMPLATE = "string_template"
    SAMPLE_FROM_COLUMN_WHERE = "sample_from_column_where"
    SHIFT_TIMESTAMP = "shift_timestamp"
    SUBSTRING = "substring"
    LUHN_APPEND = "luhn_append"
    ROUND = "round"

`rockfish.actions.ent.Derivation`

Derivation specification for derived columns.

SUM derivation

Derivation(
    function_type=DerivationFunctionType.SUM,
    dependent_columns=["col1", "col2"],
    params=SumParams()
)

SAMPLE_FROM_COLUMN derivation

Derivation(
    function_type=DerivationFunctionType.SAMPLE_FROM_COLUMN,
    dependent_columns=["users.user_id"],
    params=SampleFromColumnParams(with_replacement=True, seed=42)
)

MAP_VALUES derivation

Derivation(
    function_type=DerivationFunctionType.MAP_VALUES,
    dependent_columns=["status"],
    params=MapValuesParams(
        mapping=[{"from": "active", "to": "high"}],
        default="unknown"
    )
)

Attributes:

Name	Type	Description
`function_type`	`DerivationFunctionType`	Type of derivation function to apply
`dependent_columns`	`list[str]`	List of column references this derivation depends on. Format: "column_name" for same entity, "entity.column" for cross-entity
`params`	`Union[SumParams, MultiplyParams, SampleFromColumnParams, MapValuesParams, CumulativeParams, ConditionalSampleParams, CopyParams, AggregateFromChildParams, FormatTimestampParams, StringTemplateParams, SampleFromColumnWhereParams, ShiftTimestampParams, SubstringParams, LuhnAppendParams, RoundParams]`	Typed parameters specific to the derivation function

Column reference formats (dependent_columns):

Same entity: "column_name"
Cross-entity: "entity_name.column_name"

For composite foreign keys, prefer column_type=ColumnType.FOREIGN_KEY with an entity relationship over explicit derivations; Rockfish handles the multi-column sampling that maintains referential integrity.

Derivation parameters

`rockfish.actions.ent.SumParams`

Parameters for sum derivation function. Sums multiple columns element-wise.

Create sum parameters

SumParams()

`rockfish.actions.ent.MultiplyParams`

Parameters for multiply derivation function. Multiplies multiple columns element-wise.

Create multiply parameters

MultiplyParams()

`rockfish.actions.ent.SampleFromColumnParams`

Parameters for sample from column derivation function. Commonly used for foreign keys and derived references.

Sample from column with replacement

SampleFromColumnParams(with_replacement=True, seed=42)

Attributes:

Name	Type	Description
`with_replacement`	`bool`	If True, allow repeated values; if False, sample without replacement. Default: `True`
`seed`	`Optional[int]`	Random seed for reproducibility. Default: `None`

`rockfish.actions.ent.ConditionalSampleParams`

Parameters for the conditional-sample derivation function: sample a column from a distribution chosen by another column's value.

For each row, the value of the single dependent (condition) column selects a sub-distribution from cases (falling back to default), and one value is drawn from it. This models a distribution that varies by category (e.g. an amount whose spread depends on a status column) as a mixture indexed by another column rather than a single marginal.

Spread conditioned on a status column

ConditionalSampleParams(
    cases={
        "healthy": Domain(type=DomainType.NORMAL_DIST,
                          params=NormalDistParams(mean=10.0, std=1.0)),
        "broken": Domain(type=DomainType.NORMAL_DIST,
                         params=NormalDistParams(mean=100.0, std=20.0)),
    }
)

Attributes:

Name	Type	Description
`cases`	`dict[Any, Domain]`	Mapping from a condition-column value to the `Domain` to sample from when the condition equals that value. Keys must match the condition column's generated values; when authored as JSON they are strings and are matched to the condition value by string representation.
`default`	`Optional[Domain]`	`Domain` used when a row's condition value is not in `cases`; if omitted, an unmatched value raises at generation time. Default: `None`

`rockfish.actions.ent.MapValuesParams`

Parameters for map values derivation function. Maps values from one or more source columns to new values using mapping rules.

Map categorical values

MapValuesParams(
    mapping=[
        {"from": "active", "to": "high"},
        {"from": "idle", "to": "low"}
    ],
    default="unknown"
)

Attributes:

Name	Type	Description
`mapping`	`list[dict[str, Any]]`	List of mapping rules, each a dict with "from" and "to" keys. "from" can be str (single column) or list[str] (tuple mapping), "to" is the mapped value
`default`	`Any`	Default value for unmapped entries. Default: `None`

Mapping rule format:

{
    "from": "source_value",  # or ["value1", "value2"] for tuple mapping
    "to": "mapped_value"
}

`rockfish.actions.ent.CopyParams`

Parameters for the copy derivation function. Mirrors its single dependent column verbatim (a row-aligned alias).

Create copy parameters

CopyParams()

`rockfish.actions.ent.CumulativeParams`

Parameters for the cumulative derivation function: a running aggregate of a measurement within each session, in timestamp order.

Used for ledger/odometer/counter columns (e.g. a cumulative event count). The derivation is session-aware: it resets at each session boundary and accumulates rows in their timestamp order, so it is only valid on a measurement column of a timestamped entity with exactly one dependent column.

Running total of per-window counts

CumulativeParams(operation="sum")

Attributes:

Name	Type	Description
`operation`	`Literal['sum', 'min', 'max']`	The running aggregate to apply, one of `"sum"` (running total), `"max"` (running peak), or `"min"` (running floor). Default: `"sum"`

`rockfish.actions.ent.AggregateFromChildParams`

Parameters for the aggregate-from-child derivation function: aggregate a child entity's rows back up onto a parent column.

For each parent row, the child rows that join to it (via the parent->child EntityRelationship.join_columns) are aggregated: count of them, or sum of one child column. An optional equality filter restricts which child rows count. When cumulative is set, the per-row aggregate is then summed within each session in timestamp order, turning a per-window total into a running counter (so a parent counter equals the matching child events by construction). Examples: order.line_count = count(line_items), order.total = sum(line_items.amount).

Count child rows of a given type per parent

AggregateFromChildParams(
    child_entity="line_items",
    operation="count",
    filter_column="kind",
    filter_value="return",
)

Attributes:

Name	Type	Description
`child_entity`	`str`	The child entity to aggregate. A parent->child relationship to it must exist.
`operation`	`Literal['count', 'sum']`	`"count"` (number of joined child rows) or `"sum"` (total of `value_column` over them). Default: `"count"`
`value_column`	`Optional[str]`	Child column to sum; required for `"sum"`, ignored for `"count"`. Default: `None`
`filter_column`	`Optional[str]`	Optional child column to filter on by equality. Default: `None`
`filter_value`	`Optional[Any]`	Value `filter_column` must equal for a child row to be counted; required when `filter_column` is set. Default: `None`
`cumulative`	`bool`	If true, accumulate the per-row aggregate within each session in timestamp order (the parent must be a timestamped entity). Default: `False`

`rockfish.actions.ent.FormatTimestampParams`

Parameters for the format-timestamp derivation function: re-express the single dependent timestamp column.

The source column is parsed as a timestamp (UTC) and re-emitted as output: "epoch_s" integer seconds, "epoch_ms" integer milliseconds, or "strftime" a string formatted with format_str. format_str is required only for "strftime".

Epoch milliseconds

FormatTimestampParams(output="epoch_ms")

Custom strftime string

FormatTimestampParams(output="strftime", format_str="%Y-%m-%dT%H:%M:%SZ")

Attributes:

Name	Type	Description
`output`	`Literal['epoch_s', 'epoch_ms', 'strftime']`	One of `"epoch_s"`, `"epoch_ms"`, or `"strftime"`. Default: `"epoch_s"`
`format_str`	`Optional[str]`	`strftime` format string, required when `output="strftime"`. Default: `None`

`rockfish.actions.ent.StringTemplateParams`

Parameters for the string-template derivation function: render a per-row string from a str.format template over the dependent columns.

template uses named fields that must each be one of the derivation's dependent_columns (e.g. "{client_mac}-{ts}.pcap"); the field-vs-column check is enforced on the Derivation.

Compose a filename from two columns

StringTemplateParams(template="{client_mac}-{ts}.pcap")

Attributes:

Name	Type	Description
`template`	`str`	`str.format` template; every named field must appear in the derivation's `dependent_columns`.

`rockfish.actions.ent.SampleFromColumnWhereParams`

Sample a value from another entity's column, filtered by an equality match.

For each row, the value is drawn from source_value_column restricted to the source rows whose source_filter_column equals this row's local filter column (the single dependent_columns entry). Models a correlated foreign key (an order picks an address only among the addresses owned by its own customer).

Attributes:

Name	Type	Description
`source_value_column`	`str`	`"entity.column"` to sample the value from.
`source_filter_column`	`str`	`"entity.column"` (same source entity) matched against the row's local filter value.
`seed`	`Optional[int]`	RNG seed for the per-group draw.

`rockfish.actions.ent.RoundParams`

Round one numeric column to a fixed number of decimal places.

Use it to land a continuous distribution on realistic units, e.g. money to two decimals (cents).

Attributes:

Name	Type	Description
`decimals`	`int`	number of decimal places to keep (>= 0).

`rockfish.actions.ent.ShiftTimestampParams`

Shift a base timestamp by a per-row offset (in unit), yielding a later time.

Reads two same-entity columns [base_timestamp, offset] and returns base + offset, so a lifecycle time follows another with a realistic, ordered gap.

Attributes:

Name	Type	Description
`unit`	`str`	the offset column's unit: `"seconds"`, `"minutes"`, `"hours"`, `"days"` or `"ms"`.

`rockfish.actions.ent.SubstringParams`

Take a fixed-position substring value[start:start+length] of one string column.

Attributes:

Name	Type	Description
`start`	`int`	0-based start index (>= 0).
`length`	`Optional[int]`	number of characters (None means to the end).

`rockfish.actions.ent.LuhnAppendParams`

Append a Luhn (mod-10) check digit to one numeric-string column.

Returns each value with its Luhn check digit appended, so the result passes Luhn validation. Useful for check-digited identifiers (payment cards, IMEIs, national IDs). Non-digit characters are ignored when computing the digit but preserved.

Complete Examples

Example 1: E-commerce Session Data

Generate user sessions with state machine transitions:

import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    IDParams,
    NormalDistParams,
    Timestamp,
    GlobalTimestamp,
    Derivation,
    DerivationFunctionType,
    SampleFromColumnParams,
    StateMachineParams,
    Transition,
    EntityRelationship,
    EntityRelationshipType,
)

schema = DataSchema(
    entities=[
        Entity(
            name="users",
            cardinality=20,
            columns=[
                Column(
                    name="user_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="USER_{id}")
                    )
                ),
                Column(
                    name="age",
                    data_type="int64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.NORMAL_DIST,
                        params=NormalDistParams(mean=35.0, std=10.0)
                    )
                )
            ]
        ),
        Entity(
            name="sessions",
            cardinality=100,
            timestamp=Timestamp(column_name="timestamp"),
            columns=[
                Column(
                    name="session_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="SESSION_{id}")
                    )
                ),
                Column(
                    name="user_id",
                    data_type="string",
                    column_type=ColumnType.DERIVED,
                    column_category_type=ColumnCategoryType.METADATA,
                    derivation=Derivation(
                        function_type=DerivationFunctionType.SAMPLE_FROM_COLUMN,
                        dependent_columns=["users.user_id"],
                        params=SampleFromColumnParams(with_replacement=True, seed=42)
                    )
                ),
                Column(
                    name="page",
                    data_type="string",
                    column_type=ColumnType.STATEFUL,
                    column_category_type=ColumnCategoryType.MEASUREMENT,
                    domain=Domain(
                        type=DomainType.STATE_MACHINE,
                        params=StateMachineParams(
                            trigger_column_name="action",
                            initial_state="homepage",
                            states=["homepage", "search", "product", "cart", "checkout", "exit"],
                            terminal_states=["exit"],
                            transitions=[
                                Transition(trigger="browse", source="homepage", dest="search", probability=0.6),
                                Transition(trigger="view_product", source="homepage", dest="product", probability=0.3),
                                Transition(trigger="leave", source="homepage", dest="exit", probability=0.1),
                                Transition(trigger="add_to_cart", source="product", dest="cart", probability=0.3),
                                Transition(trigger="back", source="product", dest="search", probability=0.5),
                                Transition(trigger="checkout", source="cart", dest="checkout", probability=0.6),
                                Transition(trigger="complete", source="checkout", dest="exit", probability=0.8),
                            ]
                        )
                    )
                )
            ]
        )
    ],
    entity_relationships=[
        EntityRelationship(
            parent_entity="users",
            child_entity="sessions",
            relationship_type=EntityRelationshipType.ONE_TO_MANY,
            join_columns={"user_id": "user_id"}
        )
    ],
    global_timestamp=GlobalTimestamp(
        t_start="2025-01-01T00:00:00Z",
        t_end="2025-01-01T06:00:00Z",
        time_interval="5min"
    )
)

action = ra.GenerateFromDataSchema(
    schema=schema,
    entity_labels={
        "users": {"use_for": "testing"},
        "sessions": {"use_for": "testing", "domain": "retail"}
    }
)

Example 2: IoT Device Monitoring

Generate device metrics with timeseries data:

import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    IDParams,
    CategoricalParams,
    TimeseriesParams,
    Timestamp,
    GlobalTimestamp,
)

schema = DataSchema(
    entities=[
        Entity(
            name="devices",
            cardinality=10,
            timestamp=Timestamp(column_name="timestamp"),
            columns=[
                Column(
                    name="device_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="DEV_{id}")
                    )
                ),
                Column(
                    name="location",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.CATEGORICAL,
                        params=CategoricalParams(
                            values=["datacenter-1", "datacenter-2", "datacenter-3"],
                            with_replacement=True
                        )
                    )
                ),
                Column(
                    name="cpu_usage",
                    data_type="float64",
                    column_type=ColumnType.STATEFUL,
                    column_category_type=ColumnCategoryType.MEASUREMENT,
                    domain=Domain(
                        type=DomainType.TIMESERIES,
                        params=TimeseriesParams(
                            base_value=50.0,
                            min_value=10.0,
                            max_value=95.0,
                            seasonality_type="peak_offpeak",
                            peak_start_hour=8,
                            peak_end_hour=22,
                            seasonality_strength=0.4,
                            noise_level=0.15,
                            spike_probability=0.05,
                            spike_magnitude=0.3
                        )
                    )
                ),
                Column(
                    name="memory_usage",
                    data_type="float64",
                    column_type=ColumnType.STATEFUL,
                    column_category_type=ColumnCategoryType.MEASUREMENT,
                    domain=Domain(
                        type=DomainType.TIMESERIES,
                        params=TimeseriesParams(
                            base_value=60.0,
                            min_value=20.0,
                            max_value=90.0,
                            seasonality_type="symmetric",
                            seasonality_strength=0.3,
                            noise_level=0.1
                        )
                    )
                )
            ]
        )
    ],
    global_timestamp=GlobalTimestamp(
        t_start="2025-01-01T00:00:00Z",
        t_end="2025-01-02T00:00:00Z",
        time_interval="15min"
    )
)

action = ra.GenerateFromDataSchema(
    schema=schema,
    entity_labels={"devices": {"device_type": "iot"}}
)

Example 3: Composite Foreign Keys

Generate data with multi-column relationships:

import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    IDParams,
    CategoricalParams,
    EntityRelationship,
    EntityRelationshipType,
)

schema = DataSchema(
    entities=[
        Entity(
            name="transport_interfaces",
            cardinality=50,
            columns=[
                Column(
                    name="device_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.CATEGORICAL,
                        params=CategoricalParams(
                            values=["DEV_1", "DEV_2", "DEV_3"],
                            with_replacement=True
                        )
                    )
                ),
                Column(
                    name="interface_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="IF_{id}")
                    )
                ),
                Column(
                    name="bandwidth",
                    data_type="int64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.CATEGORICAL,
                        params=CategoricalParams(
                            values=[1000, 10000, 100000],
                            with_replacement=True
                        )
                    )
                )
            ]
        ),
        Entity(
            name="cell_sites",
            cardinality=200,
            columns=[
                Column(
                    name="site_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="SITE_{id}")
                    )
                ),
                # Composite foreign key - both columns marked as foreign_key
                Column(
                    name="transport_device_id",
                    data_type="string",
                    column_type=ColumnType.FOREIGN_KEY,
                    column_category_type=ColumnCategoryType.METADATA
                ),
                Column(
                    name="transport_interface_id",
                    data_type="string",
                    column_type=ColumnType.FOREIGN_KEY,
                    column_category_type=ColumnCategoryType.METADATA
                )
            ]
        )
    ],
    entity_relationships=[
        EntityRelationship(
            parent_entity="transport_interfaces",
            child_entity="cell_sites",
            relationship_type=EntityRelationshipType.ONE_TO_MANY,
            join_columns={
                "device_id": "transport_device_id",
                "interface_id": "transport_interface_id"
            }
        )
    ]
)

action = ra.GenerateFromDataSchema(schema=schema)

Example 4: Realistic Customer Profiles

Combine globally-unique keys, realistic named entities, a skewed numeric distribution, and a multimodal column, with a global seed for reproducibility:

import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    UniqueParams,
    NamedEntityProvider,
    NamedEntityProviderParams,
    CategoricalParams,
    LogNormalDistParams,
    MixtureParams,
)

schema = DataSchema(
    entities=[
        Entity(
            name="customers",
            cardinality=1000,
            columns=[
                # Globally-unique account key, e.g. "ACCT-000001"
                Column(
                    name="account_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.UNIQUE,
                        params=UniqueParams(format="template", template_str="ACCT-{index:06d}")
                    )
                ),
                # Realistic full names
                Column(
                    name="name",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.NAMED_ENTITY_PROVIDER,
                        params=NamedEntityProviderParams(
                            provider=NamedEntityProvider.PERSON_FULL_NAME,
                            locale="en",
                        )
                    )
                ),
                # Unique-ish emails drawn without replacement from a large pool
                Column(
                    name="email",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.NAMED_ENTITY_PROVIDER,
                        params=NamedEntityProviderParams(
                            provider=NamedEntityProvider.PERSON_EMAIL,
                            unique_values=5000,
                            with_replacement=False,
                        )
                    )
                ),
                Column(
                    name="plan",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.CATEGORICAL,
                        params=CategoricalParams(
                            values=["free", "pro", "enterprise"],
                            weights=[0.7, 0.25, 0.05],
                        )
                    )
                ),
                # Annual income: right-skewed, spanning orders of magnitude
                Column(
                    name="annual_income",
                    data_type="float64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.LOGNORMAL_DIST,
                        params=LogNormalDistParams(mu=11.0, sigma=0.5)
                    )
                ),
                # Monthly spend: a mixture of churned (0) and active (lognormal) customers
                Column(
                    name="monthly_spend",
                    data_type="float64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.MIXTURE,
                        params=MixtureParams(
                            components=[
                                {
                                    "weight": 0.3,
                                    "domain": Domain(
                                        type=DomainType.CATEGORICAL,
                                        params=CategoricalParams(values=[0.0]),
                                    ),
                                },
                                {
                                    "weight": 0.7,
                                    "domain": Domain(
                                        type=DomainType.LOGNORMAL_DIST,
                                        params=LogNormalDistParams(mu=4.0, sigma=0.6),
                                    ),
                                },
                            ]
                        )
                    )
                ),
            ]
        )
    ],
    seed=42,
)

action = ra.GenerateFromDataSchema(
    schema=schema,
    entity_labels={"customers": {"use_for": "demo"}},
)

Example 5: Hierarchical Fan-out (Orders and Line Items)

Use count-driven fan-out so each order explodes into a variable number of line items, number each line within its order with GROUP_ORDINAL, and roll the line totals back up onto the order with AGGREGATE_FROM_CHILD:

import rockfish.actions as ra
from rockfish.actions.ent import (
    DataSchema,
    Entity,
    Column,
    ColumnType,
    ColumnCategoryType,
    Domain,
    DomainType,
    IDParams,
    CategoricalParams,
    GroupOrdinalParams,
    NormalDistParams,
    Derivation,
    DerivationFunctionType,
    AggregateFromChildParams,
    EntityRelationship,
    EntityRelationshipType,
)

schema = DataSchema(
    entities=[
        Entity(
            name="orders",
            cardinality=100,
            columns=[
                Column(
                    name="order_id",
                    data_type="string",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.ID,
                        params=IDParams(template_str="ORD_{id}")
                    )
                ),
                # Per-order line count: drives how many line_items rows each order gets
                Column(
                    name="line_count",
                    data_type="int64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.CATEGORICAL,
                        params=CategoricalParams(values=[1, 2, 3, 4, 5])
                    )
                ),
                # Order total: sum of the child line-item amounts
                Column(
                    name="order_total",
                    data_type="float64",
                    column_type=ColumnType.DERIVED,
                    column_category_type=ColumnCategoryType.METADATA,
                    derivation=Derivation(
                        function_type=DerivationFunctionType.AGGREGATE_FROM_CHILD,
                        dependent_columns=[],
                        params=AggregateFromChildParams(
                            child_entity="line_items",
                            operation="sum",
                            value_column="amount",
                        )
                    )
                ),
            ]
        ),
        Entity(
            name="line_items",
            cardinality=1,  # ignored: row count is driven by orders.line_count
            columns=[
                # Foreign key back to the order; filled in by the relationship
                Column(
                    name="order_id",
                    data_type="string",
                    column_type=ColumnType.FOREIGN_KEY,
                    column_category_type=ColumnCategoryType.METADATA
                ),
                # 0-based line number within each order
                Column(
                    name="line_no",
                    data_type="int64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.GROUP_ORDINAL,
                        params=GroupOrdinalParams(start=0)
                    )
                ),
                Column(
                    name="amount",
                    data_type="float64",
                    column_type=ColumnType.INDEPENDENT,
                    column_category_type=ColumnCategoryType.METADATA,
                    domain=Domain(
                        type=DomainType.NORMAL_DIST,
                        params=NormalDistParams(mean=40.0, std=12.0)
                    )
                ),
            ]
        ),
    ],
    entity_relationships=[
        EntityRelationship(
            parent_entity="orders",
            child_entity="line_items",
            relationship_type=EntityRelationshipType.ONE_TO_MANY,
            join_columns={"order_id": "order_id"},
            child_count_column="line_count",
        )
    ],
    seed=7,
)

action = ra.GenerateFromDataSchema(schema=schema)

Tips and Best Practices

Schema Design

Start simple: Begin with metadata-only entities, then add measurements and relationships.
Validate incrementally: Build your schema step by step to catch validation errors early.
Use meaningful names: Entity and column names should reflect the domain you are modeling.
Use typed objects: Prefer the typed Python classes over raw dicts for better type checking and IDE support.
Set a seed when it matters: Add DataSchema.seed for demos, fixtures, and regression tests that must be byte-for-byte reproducible; leave it unset when you want fresh variation each run (the run seed is still logged and labeled so you can reproduce it later).

Choosing a domain

Identifiers: ID for templated keys tied to row position, UNIQUE for guaranteed-unique keys, MAC addresses, or IPs, and GROUP_ORDINAL to number child rows within a parent.
Categorical vs. named entities: Use CATEGORICAL for a small fixed set of choices; use NAMED_ENTITY_PROVIDER for realistic names, emails, addresses, and similar fields drawn from a large pool.
Distributions: NORMAL_DIST for symmetric quantities, LOGNORMAL_DIST for right-skewed quantities that span orders of magnitude (latencies, incomes, sizes), EXPONENTIAL_DIST for waiting times, UNIFORM_DIST for bounded ranges.
Multimodal columns: Use MIXTURE to blend simpler domains (for example a point mass at zero plus a heavy tail).

Column and relationship types

Independent columns are best for entity IDs, static attributes (age, name, category), and random categorical values.
Stateful columns are best for time-varying measurements (CPU usage, temperature) and behavioral patterns (page navigation, transaction flows).
Derived columns are best for computed values (totals, aggregations), mapped or reformatted values, and cross-entity roll-ups.
Composite foreign keys: Mark all foreign-key columns as FOREIGN_KEY and define the multi-column relationship in entity_relationships.
Hierarchical fan-out: Use a relationship's child_count_column to expand each parent row into a variable number of child rows, then pair it with GROUP_ORDINAL and AGGREGATE_FROM_CHILD so parent counters and totals match the child rows by construction.
Realistic timeseries: Use TIMESERIES columns with peak_offpeak seasonality and tune noise and spikes for anomalies. ```