In this tutorial, you'll discover how to leverage Bagel's APIs to fine-tune AI models.
Step 1: Preparing Your RAW Asset
Before we can fine-tune a model, you need to have your Asset ready within Bagel. If you haven't already, use this code to create your Asset:
import { Settings, Client } from'bagelml'// Settings configconstsettings=newSettings({ bagel_api_impl:"rest", bagel_server_host:"api.bageldb.ai",})// Create Bagel clientconstclient=newClient(settings)constapi_key="insert api key"constpayload= { dataset_type:"input dataset type",// e.g "RAW", "VECTOR", "MODEL" title:"Input title",// e.g "Apple" category:"inpute category",// e.g "Cat1", "Cat2" details:"input details",// e.g "testing" tags: ["Optional input"],// e.g ["AI", "DEMO", "TEST"] or [ ] user_id:"input user id"// e.g "345281182308180743453"}// Calling the create asset methodconstcreateAsset=async () => {// get response of created assetconstres=awaitclient.create_asset(payload, apiKey)console.log(res)}createAsset()
Create a MODEL
// Define the payload for creating a dataset. Just replace the payload above with this oneconstpayload= { dataset_type:"MODEL", title:"Python testing Create MODEL", category:"Cat1", details:"Testing", tags: ["AI","DEMO","TEST"], user_id:"101481188466180740994"}
Upload file to RAW Asset
Navigate to your console and select my datasets on the left side of your page. Select the RAW asset you created and navigate to the files tab. Add a .txt file of your choice. You can as well run this code to upload file from your computer:
With our Assets in place, we can now leverage Bagel's fine-tuning capabilities to train a custom model. The POST /api/v1/asset endpoint allows us to initiate a fine-tuning job for a pre-trained model using our Asset:
import { Settings, Client } from'bagelml'// Settings configconstsettings=newSettings({ bagel_api_impl:'rest', bagel_server_host:'api.bageldb.ai',})constclient=newClient(settings)constapiKey='input your API key',// e.g 'm34282rbS6g86xSqPhi27fjfZqEc2ErkloJ'const payload = { dataset_type:'MODEL', title:'model1', category:'CAT1', details:'test', tags: [], user_id:'9a198051-7fae-4e06-81aa-dcfc988d4d0f', fine_tune_payload: { asset_id:'bc0f4e4e-d468-4c07-8db1-1fc66873c682',// Move asset_id here model_name:'model1',// Same as the title base_model:'b0cb6e74-5f4a-4af1-ac5e-872ef3962b1c', file_name:'test.txt', user_id:'9a198051-7fae-4e06-81aa-dcfc988d4d0f', },}// Calling the fine tune methodconstfineTune=async () => {constasset=awaitclient.fine_tune(payload, apiKey)console.log(asset)}fineTune()
Once the fine-tuning job is started, Bagel will return a asset_idof the fine-tuned model. We can use this to retrieve a job id which will allow us monitor the progress of the fine-tuning process.
Step 3: Monitoring the Fine-Tuning Job - Getting Job by Asset
import { Settings, Client } from'bagelml'// Settings configconstsettings=newSettings({ bagel_api_impl:'rest', bagel_server_host:'api.bageldb.ai',})constclient=newClient(settings)constapi_key="input API key"// e.g "67YUtJPByf8qnUihA388378GYsZ766HG"constasset_id="input dataset id"// e.g "eR57yd73b-7gt4-4a71-b0fd-afKJUIY6578Ge"// Calling the get job by asset id methodconstgetJobByAsset=async () => {constjob=awaitclient.get_job_by_asset(asset_id, apiKey)console.log(job)}getJobByAsset()
In this example, we're sending a GET request to the /api/v1/asset/{asset_id} endpoint. The response contains information about the job id.
Step 4: Monitoring the Fine-Tuning Job - Get Job
import { Settings, Client } from'bagelml'// Settings configconstsettings=newSettings({ bagel_api_impl:'rest', bagel_server_host:'api.bageldb.ai',})constclient=newClient(settings)job_id ="input job_id" # Replace with the actual job ID. e.g "8566657552882860032"api_key ="input API key" # Replace with your actual APIkey. e.g "67YUtJPByf8qnUihA388378GYsZ766HG"// Calling the get job method using job idconstgetJob=async () => {constresult=awaitclient.get_job(job_id, apiKey)console.log(result)}getJob()
In this example, we're sending a GET request to the /api/v1/jobs/{job_id} endpoint, providing the job_id as a path parameter. The response contains information about the job, including its status (e.g., "running", "completed", "failed") and progress (e.g., a percentage value indicating how much of the job has been completed).
You can periodically check the job status and progress until the fine-tuning process is complete.
List Jobs
You can also list all jobs using this code:
import {Settings, Client} from'bagelml'server_settings =Settings( bagel_api_impl ="rest", bagel_server_host ="api.bageldb.ai",)constclient=newClient(server_settings)// Replace "your_api_key_here" with the provided API keyconstapi_key="input API key" # Replace with your actual APIkey. e.g "67YUtJPByf8qnUihA388378GYsZ766HG"constuser_id="input your user id" # e.g "231243124466184567909"// Call the list jobs methodconstlistJob=async () => {constres=awaitclient.list_jobs(user_id, api_key)console.log(res)}listJob()
Step 5: Accessing the Fine-Tuned Model
Once the fine-tuning job is complete, you can access the fine-tuned model and its associated files using the GET /api/v1/jobs/asset/{asset_id}/files/{file_name}
In this example, we're using the GET /api/v1/jobs/asset/{asset_id}/files/{file_name} endpoint to download a specific file. You can replace file_name with the appropriate file name for your use case.