About
Prototype and deploy open source LLMs
Migrating to open-source LLMs
Whether you’re starting with open-source LLMs from scratch or moving away from closed source LLMs,
it can be a stressful and unclear process. If you’re looking to switch over to open source models,
prompt re-engineering and evaluating the quality of the resulting responses are important
considerations.
Use our prototype-to-deployment stack to help make that transition easier.
Secure on-prem deployment
Alternatively, you might have strict privacy and security concerns, in which case a trusted, secure
on-prem deployment provided by nCompass is required. Before committing to such a large deployment,
you can test the base quality of various open-source models using our
API.
Prototype
Use our OpenAI compatible API to help you decide which open source model is a viable equivalent to your current closed-source LLM.
Deploy
Once you’ve made up your mind on which model(s) to use, simply use nCompass to setup your deployment.
OpenAI Compatible API
If you want to deploy with our OpenAI compatible API, all you have to do is provide us with
your nCompass authentication details and follow the
API setup.
Secure On-prem Deployment
If you want to deploy your own secure on-prem deployment, all you have to do is sign-up
with us and we will deal with:
- Launching instances
- Autoscaling based on your workload
- Accelerating your selected models
Our system optimizes for GPU utilization so you can sleep well knowing that your deployment costs are being minimized.
Please contact us for pricing and information regarding deployment.