Skip to main content

Quick Start

The Bundling Service helps users create and manage model bundles for Liquid Edge AI Platform (LEAP). Currently users interact with it through leap-bundle, a command-line interface (CLI).

The CLI supports two inference engines for model bundling:

  • GGUF (default): Generates .gguf files for llama.cpp inference
  • ExecuTorch (deprecated): Generates .bundle files for ExecuTorch inference (use --executorch flag). This option may be removed in a future version.

The CLI also supports downloading GGUF models directly from JSON manifest files.

This guide will help you quickly get started with both features.

Install​

pip install leap-bundle

Quick Start for GGUF Model Download​

leap-bundle download <model-name> [--quantization <quantization>]

Example:

leap-bundle download LFM2-1.2B --quantization Q5_K_M

The command will:

  1. Resolve the appropriate manifest URL for the model/quantization.
  2. Create an output directory based on the URL or according to --output-path if specified.
  3. Download the JSON manifest file for the given combination of model and quantization, if it exists.
  4. Download all model files referenced in the manifest.
  5. Update the local manifest to use relative paths to the downloaded files.
tip

Manifest downloads don't require authentication with leap-bundle login. They work immediately after installation.

Quick Start for Model Bundling​

info

Compatability Note: The Model Bundling Service will work with any fine-tuned model, as long as the model architecture comes from a base model that is part of the LEAP model library.

If you have a custom-trained or fine-tuned model, you can create a model bundle for use with LEAP SDK. By default, the CLI generates GGUF files for llama.cpp inference. Use the --executorch flag to generate ExecuTorch bundles instead.

Authenticate​

  1. Sign in on the LEAP website

  2. Click the account icon on the top right, and go to your profile

  3. Select the API keys tab and create a new API key

api-key-screenshot

  1. Authenticate the Model Bundling Service with your API token:
leap-bundle login <api-key>

Example output:

ℹ Validating API token...
✓ Successfully logged in to LEAP platform!

Create model bundle​

  1. Prepare your model checkpoint.
  2. Create a bundle request:
leap-bundle create <path-to-your-model-checkpoint>

Example output:

ℹ Calculating directory hash...
ℹ Submitting bundle request...
✓ Bundle request created with ID: 1
ℹ Starting upload...
Uploading directory... ✓
✓ Upload completed successfully! Request ID: 1
  1. Check request status:
leap-bundle list

Example output:

Bundle Requests (50 most recent)
┏━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID   ┃ Input Path                      ┃ Status       ┃ Creation               ┃ Notes                       ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1    │ /path/to/your/model/directory   │ processing   │ 2024-01-15T10:30:00Z   │ Request is being processed. │
└──────┴─────────────────────────────────┴──────────────┴────────────────────────┴─────────────────────────────┘
✓ Found 1 bundle requests.

Get details for a specific request:

leap-bundle list <request-id>

Example output:

✓ Request Details:
  ID:         1
  Input Path: /path/to/your/model/directory
  Status:     completed
  Creation:   2024-01-15T10:30:00Z
  Update:     2024-01-15T10:45:00Z
  Notes:
  1. When the request is Completed, you can download the bundle:
leap-bundle download <request-id>

Example output:

ℹ Requesting download for bundle request 1...
✓ Download URL obtained for request 1
Downloading bundle output... ✓
✓ Download completed successfully! File saved to: model-Q4_K_M.gguf

The model files will be saved in the current directory. GGUF bundling produces .gguf files, while ExecuTorch bundling produces .bundle files.

Complete Example​

Here's a complete example showing the full workflow:

# 1. Install and authenticate
pip install leap-bundle
leap-bundle login <api-key>
leap-bundle whoami

# 2. Create a bundle request (GGUF by default)
leap-bundle create <model-directory>

# Or create an ExecuTorch bundle
leap-bundle create <model-directory> --executorch

# 3. Monitor the request (repeat until completed)
leap-bundle list

# 4. Download when ready
leap-bundle download <request-id>

# 5. Your model files are now ready to use!
ls -la <downloaded-model-files>

Managing Requests​

Cancel a Request​

If you need to cancel a request that's still processing:

leap-bundle cancel <request-id>

Example output:

ℹ Cancelling bundle request 1...
✓ Request cancelled successfully.

Log out​

You can log out to clear your stored credentials:

leap-bundle logout

Example output:

✓ Successfully logged out from LEAP platform!

Next Steps​