rockfish

Create a new local Connection.

A local connection executes locally on the machine and not on the Rockfish API.

Create a new remote Connection.

The connection should be used as a context manager when possible in order to close and free up resources:

async with rf.Connection.remote("MY_API_KEY") as conn:
    ...

As context managers cannot span multiple cells in a notebook, you can create an instance and close it manually:

conn = rf.Connection.remote("MY_API_KEY")
...
await conn.close()

Use the api_url, project, and organization keyword arguments to use a non-default value.

Create a new remote Connection using a Rockfish config file.

The Rockfish config file contains one or more profiles that can be selected from by name to use the api_key, api_url, and optionally a project and organization.

A Rockfish config file can be created in the user's home directory at ~/.config/rockfish/config.toml. If it does not exist, it can be created manually.

It is recommended to set the permissions of this file to allow only yourself to read it:

chmod 600 ~/.config/rockfish/config.toml

The config uses the TOML format.

Example:

[profile.default]
api_url = "https://api.rockfish.ai"
api_key = "your api key..."

[profile.ocp]
api_url = "https://api.rockfish.ai"
api_key = "your api key..."
organization = "7sEeE4FFWhps2mt0uy1iln"

Create a new Connection based on environment variables.

The following environment variables can be set:

• ROCKFISH_API_KEY (required)
• ROCKFISH_API_URL
• ROCKFISH_PROJECT_ID
• ROCKFISH_ORGANIZATION_ID

Close the Connection, freeing all resources.

>>> workflow = await conn.close()

Get the active Organization.

>>> await conn.active_organization()

Get a Stream of Organizations.

>>> async for org in conn.organizations()
...     print(org)

Get the active Project.

>>> await conn.active_project()

Get a Stream of Projects.

>>> async for project in conn.projects()
...     print(project)

Parameters:

Create a new Project.

>>> await conn.create_project("myproject")

Change the active project to project, all further calls on the Connection will be with this project.

>>> conn.change_project("42uy7DKYiwYx4cWJt5Xo2R")

>>> project = await conn.projects(name="myproject").nth(0)
>>> conn.change_project(project)

Get a Stream of Users in your organization.

>>> async for user in conn.users()
...     print(user)

Get a User from a user_id.

Get your current active User.

Get a Stream of your Tokens.

>>> async for token in conn.tokens()
...     print(token)

Create a new Token for your user.

>>> await conn.create_token()

Delete a Token.

>>> await conn.delete_token(token)

Get a Stream of Workflows.

>>> async for workflow in conn.workflow():
...     print(workflow)

>>> async for workflow in conn.workflow(
        status=rf.Status.COMPLETED,
        labels={"project": "icarus"},
        after=datetime.now(timezone.utc) - timedelta(days=20),
        order=Order.DESCENDING,
        limit=20,
    ):
...     print(workflow)

Parameters:

Get a Workflow by ID.

>>> workflow = await conn.get_workflow("5X2M615Ot0SzoAh3mN5sto")

Get a Stream of datasets.

>>> async for dataset in conn.datasets():
        print(dataset)

Create a new LocalDataset from a query.

>>> await conn.query_datasets('SELECT * FROM "2C4BUEiZTUJaTALGdim2oX";')

Get a Stream of Models.

>>> async for models in conn.models():
        print(models)

Get a Model by ID.

>>> model = await conn.get_model("6zMYbFK2p3YwSEUR2oaBc8")

Add a dataset to the Workflow.

Add a model to the Workflow.

Add an action to the Workflow.

Add an action, model or dataset to the Workflow.

Add a path of action, model or dataset to the Workflow.

The items are connected from left to right, parent to child.

Require the workflow to run on a non-system worker group.

Start the workflow running on the conn.

Returns the workflow ID, an opaque string that uniquely identifies the workflow.

>>> workflow.id()
2ZxiQe4SDTNYYSjd87Wshp

Return a string describing the Workflow status.

Request the workflow to stop.

>>> await workflow.stop()

Wait until the workflow is completed.

>>> await workflow.wait()

Return a Stream of the workflow raw events.

>>> async for event in workflow.events():
...     print(event)

Parameters:

Return a Stream of the workflow StateEvent.

>>> async for state in workflow.states():
...     print(state)

Parameters:

Return a Stream of the workflow LogEvent.

>>> async for log in workflow.logs():
...     print(log)

Parameters:

Return a Stream of the workflow LinkEvent.

>>> async for links in workflow.links():
...     print(links)

Parameters:

Return a Stream of the workflow ProgressEvent.

>>> async for progress in workflow.progress():
...     print(progress)

Parameters:

Return a Stream of the workflow RemoteDataset.

>>> async for dataset in workflow.datasets():
...     print(dataset)

Parameters:

Return the first model_id.

Deprecated: Use AbstractWorkflow.models().

Return a Stream of the workflow Model.

>>> async for model in workflow.models():
...     print(model)

Create a RemoteDataset from the Dataset ID.

Create a LocalDataset from the contents of a CSV file.

Create a LocalDataset from the contents of a Parquet file.

Create a LocalDataset from the contents of a Pandas dataframe.

Create a LocalDataset from an Arrow table.

Create a LocalDataset from the contents of a JSON file.

Create a LocalDataset with name and the pyarrow.Table table as the contents.

Create a LocalDataset with name and the contents contained in the csv encoded file at path.

Parameters:

Create a LocalDataset from the contents of a JSON file, named name and read from json_file_path. Currently supported JSON formats: .json and .jsonl [newline-delimited JSON].

e.g. valid JSON file:

[
    {"name": "Alice", "age": 25, "address": {"city: "New York", "state": "NY"}},
    {"name": "Bob", "age": 30, "address": {"city: "San Francisco", "state": "CA"}}
]

valid JSONL file:

{"name": "Alice", "age": 25, "address": {"city: "New York", "state": "NY"}}
{"name": "Bob", "age": 30, "address": {"city: "San Francisco", "state": "CA"}}

>>> import rockfish as rf
>>> rf.Dataset.from_json('my_dataset', 'data.json')
LocalDataset('my_dataset')

Parameters:

Raises:

Return a LocalDataset containing the results of an SQL query against this Dataset.

>>> ds = await dataset.sql("select foo from my_table")

Parameters:

Return a LocalDataset containing the results of an SQL query against this Dataset.

>>> ds = dataset.sync_sql("select foo from my_table")

Parameters:

Return a new LocalDataset with the provided TableMetadata. This metadata can be accessed from the Actions within a Workflow.

Return the TableMetadata.

Return a LocalDataset containing only the specified fields.

Return a LocalDataset with the field old renamed to new.

Return a LocalDataset without the fields.

Return a LocalDataset containing only the n first records.

Convert to a LocalDataset.

Convert to a RemoteDataset.

Return a pandas dataframe corresponding to this Dataset.

Create a remote Dataset that references the id.

Raises:

Return the name of the Dataset.

Delete the remote Dataset from the server.

Return a LocalDataset containing the results of an SQL query against this Dataset.

Return a local Dataset containing the full remote Dataset.

Return the value of the secret token.

Stream of Users in the Project.

>>> async for user in project.users()
...     print(user)

Add a user to the Project.

>>> user = await organization.users().filter(
...     lambda u: u.email == "bob@example.org").nth(0)
>>> await project.add_user(user)

Remove a user from the Project.

>>> user = await project.users().filter(
...     lambda u: u.email == "bob@example.org").nth(0)
>>> await project.remove_user(user)

Stream of Users in the Organization.

>>> async for user in organization.users()
...     print(user)

Add a user to the Organization.

This method requires that you have the Admin role for the Organization.

>>> await org.add_user("3FJ5Tulq5mUXMadbk63sHO")

Remove a user from the Organization.

This method requires that you have the Admin role for the Organization.

>>> user = await org.users().filter(lambda u: u.email == "bob@example.org").nth(0)
>>> await org.remove_user(user)