The Kuality AI CLI Workflow

We’re excited to announce that Kuality AI now has a command line interface, offering a new workflow for your data quality monitoring. We wanted to provide users with another approach to their data quality that doesn’t involve navigating a user interface. We recognize that everyone has a different preference on how to perform their data quality monitoring and Kuality AI wants to provide you with as many approaches as possible for data quality monitoring.    

Initializing The Kuality AI CLI

When a user wants to use the CLI with their Kuality AI instance, they must first do a quick configuration. All they need is to have their company’s Quaytics URL and one of their personal tokens. Then the user can run the following command to have the CLI ready to go.

   kuality-ai init
             --url "https://your-kuality-ai.kuality-ai.io/"
             --token "YOUR_TOKEN_HERE"


Triggering An Operation VIA The CLI

The Kuality AI CLI now allows a user to trigger any one of our operations on a connected datastore, providing an easy way to check your company’s data quality at a moment’s notice. No longer does an operation have to be triggered through the UI, but can be done at any time through your command line. The Kuality AI CLI also offers all the same parameters as seen in our user interface. The first operation that can be triggered by the CLI is our catalog operation. During the catalog operation, the Kuality AI platform will analyze the source datastore’s metadata, preparing it for subsequent Profile and Scan operations.

   kuality-ai run catalog
                 --datastore "DATSTORE_ID_LIST"
                 --include "INCLUDE_LIST"
                 --prune
                 --recreate
                 --background


The second operation that can be triggered from the CLI is our profile operation. This operation generates valuable metadata insights for all the data assets within the source datastore. It also automatically infers customized data quality checks based on the profiled data.

   kuality-ai run profile
                 --datastore "DATSTORE_ID_LIST"
                 --container_names "CONTAINER_NAMES_LIST"
                 --container_tags "CONTAINER_TAGS_LIST"
                 --infer_constraints
                 --max_records_analyzed_per_partition "MAX_RECORDS_ANALYZED_PER_PARTITION"
                 --max_count_testing_sample "MAX_COUNT_TESTING_SAMPLE"
                 --percent_testing_threshold "PERCENT_TESTING_THRESHOLD"
                 --high_correlation_threshold "HIGH_CORRELATION_THRESHOLD"
                 --greater_then_date "GREATER_THAN_TIME"
                 --greater_than_batch "GREATER_THAN_BATCH"
                 --histogram_max_distinct_values "HISTOGRAM_MAX_DISTINCT_VALUES"
                 --background


The last operation that can be triggered from the CLI is our scan operation. When a scan is triggered on a source datastore, the Kuality AI engine asserts the automatically inferred checks (as well as any additional checks you create) against both historical and new data within the source datastore.

   kuality-ai run scan
                 --datastore "DATSTORE_ID_LIST"
                 --container_names "CONTAINER_NAMES_LIST"
                 --container_tags "CONTAINER_TAGS_LIST"
                 --incremental
                 --remediation
                 --max_records_analyzed_per_partition "MAX_RECORDS_ANALYZED_PER_PARTITION"
                 --enrichment_source_records_limit
                 --greater_then_date "GREATER_THAN_TIME"
                 --greater_than_batch "GREATER_THAN_BATCH"
                 --background


Lastly, if you want to start an operation, but not waiting for the operation to finish, to use your command line again, you can use the –background parameter to start an operation without having the command line wait for it to finish. This is especially useful because if your datastore possesses a huge quantity of data and operations take a long time to finish, your terminal will not be clogged by the Kuality AI CLI.    

Check Operation Status

The Kuality AI CLI also allows you to check the status of any operation that was triggered. If you triggered an operation to run in the background and want to see its status, just run this simple command to find out. The Kuality AI CLI will report the correct status of the triggered operation, even if the operation failed or was aborted.  

   kuality-ai operation check_status
                       --ids "OPERATION_IDS"


Export Checks

The export checks command allows you to export all the checks from a datastore. With this, you can migrate the quality checks to different datastores. Example:

   kuality-ai checks export --datastore 1 --containers 1,2


This exports checks from datastore with ID ‘1’, from containers with IDs ‘1’ and ‘2’, and saves them to $HOME/.kuality-ai/data_checks.json

Import Checks

The import checks command allows you to import data quality checks from a file into a datastore. Example:

   kuality-ai checks import --datastore 2,3


This imports checks from the default $HOME/.kuality-ai/data_checks.json into datastores with IDs ‘2’ and ‘3’

Broaden Your Data Quality Pipeline

By leveraging the Kuality AI CLI, we’re providing companies with another pipeline for their data quality governance pipeline. No longer does one have to use the user interface to accomplish their goals, but can use the Kuality AI CLI to get the same end results. At the same time, Kuality AI is committed to improving and expanding the CLI’s capabilities to provide the user with the smoothest experience in their data quality governance goals.

To learn more about the full feature list of the Kuality AI CLI, you can visit the pypi kuality-ai-cli package or our User Guide .