Skip to content

[ODSC-68321] Create grouped model with multiple verified models #1064

New issue

Have a question about this project? Sign up for a free account to open an issue and contact its maintainers and the community.

By clicking “Sign up for ”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on ? Sign in to your account

Merged

Conversation

VipulMascarenhas
Copy link
Member

Description

This PR adds the functionality to support group model creation when more than one model is selected to be deployed. The PR does not cover the deployment part, that needs to be covered by a separate PR.

Usage

The create model API is not accessible directly to the user, we need to use the create deployment API to test this.

ads aqua deployment create 
--instance_shape VM.GPU.A10.2 
--display_name "Model grouping deployment"  
--model_info '[{"model_id": "ocid1.datasciencemodel.oc1.iad.<ocid>", 
                           "gpu_count": 1}, 
                         {"model_id": "ocid1.datasciencemodel.oc1.iad.<ocid2>", 
                          "gpu_count": 1}]'

The new model is called "model_group_YYYYMMDD", and has the freeform tags

multimodel: true
OCI_AQUA: active

The custom metadata added in this model are:

deployment-container: odsc-vllm-serving
model_group_count: 2
model-id-0: ocid1.datasciencemodel.oc1.iad.<ocid>
model-name-0: tiiuae/falcon-7b
artifact_location-0: oci://service-managed-models@<namespace>/service_models/falcon-7b/898df13/artifact
model-id-1:  ocid1.datasciencemodel.oc1.iad.<ocid>
model-name-1: phi-2
artifact_location-1: oci://service-managed-models@<namespace>/service_models/phi-2/b10c3eb/artifact

@oracle-contributor-agreementoracle-contributor-agreement bot added the OCA VerifiedAll contributors have signed the Oracle Contributor Agreement.label Feb 7, 2025
@mrDzurbmrDzurb merged commit c25a3de into feature/multi_model_deployment Feb 10, 2025
1 check passed
Sign up for free to join this conversation on . Already have an account? Sign in to comment
Labels
OCA VerifiedAll contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants