Keboola Docs

CLI

Keboola CLI (Command Line Interface), known also as “Keboola as Code”, is a set of commands for operating your cloud data pipeline. It is available to install in the Windows, macOS, and Linux environments.

The whole Keboola project is represented by a local directory structure. Component configurations are represented by JSON files.

Use Cases

Keboola CLI can be used, for example, to:

Pull your entire project to a local directory in seconds. See the init and pull commands.
Bulk edit component configurations in your IDE.
Compare the local version with the current project state. See the diff command.
Copy a configuration as a directory in the project and between projects. See the persist command.
Apply all changes back to the project in a moment. See the push command.
Manage project history in a git repository.
Automate the whole process in a CI/CD pipeline. See GitHub Integration. Use the --skip-workflows flag during initialization to avoid interactive prompts in automated environments.
Merge and rebase Keboola Branches via Git. Learn more in the Example Use Cases section.
Distribute a single project definition into multiple projects. See the Example Use Cases section.
Multi-stage (and multi-project) environment management via Git. See the Example Use Cases section.
Locally develop and test your dbt transformation code.

Subsystems

A brief overview of supported subsystems of the project.

Configurations

Component configurations and configuration rows are fully supported.
This includes all special types of components, such as:
- Transformations, Variables, Shared Codes, Schedules and Orchestrations.

Development Branches

A branch can be pulled and then edited or deleted locally.
Changes can be pushed back to the project.
There is one limitation, a branch cannot be created locally.
- A branch must be created directly in the project, from the main branch.
- See the Create Branch command.

Storage

At the moment, all Storage related operations are sub-commands of the kbc remote command. They operate directly on a project. This means that any changes you make using the CLI are immediately applied to your project. We have plans to add support for managing buckets and tables locally using definition files just like component configurations.

Files

To upload a file, use the file upload command.
To download a file, use the file download command.

Buckets and tables

These commands can be used to manage the buckets and tables in your project:

To create a new bucket, use the create bucket command.
To create a new table, use the create table command.

The resulting tables will be empty, so you may want to use:

The table import command to import data.
The table unload command can be used to take data out of a table and store it in a file.

For convenience, you can use combined commands:

The table upload command combines the file upload + table import operations.
The table download command combines the table unload + file download operations.

These commands may be a little heavy if you are dealing with a lot of data.

If you just want a quick sample, use the table preview command.