We work hard to ensure that using our API is as easy as possible. Simply follow the three steps below to prototype your application using our free API.

Note that as it is a free API, your requests are queued, so some requests may take significantly longer than others. Contact us if you would like a dedicated deployment with better QoS.

1

Install the nCompass client

pip install ncompass
2

Choose your model

To use our free API, please take a look at our available models and set the NCOMPASS_API_KEY for the in the client to the appropriate value as presented there.

3

Execute

The code snippets below can be found on our github here.

The following code is an example of using the nCompass client for prompt completion.

from ncompass.client import nCompass

client = nCompass(api_key = 'llama2')
'''
Sessions are a performance optimization to maintain a link between you and the server. If you want
to run multiple prompts, call the `complete_prompt` command multiple within a started session
rather than restarting multiple sessions.
'''
client.start_session()
client.wait_until_model_running()
params = {'max_tokens':    300 # max output tokens requested
          , 'temperature': 0.5
          , 'top_p':       0.9
          , 'stream':      True}
prompt = 'Give me 5 tools I can use to accelerate inference of my ML model?'
response = client.complete_prompt(prompt, **params)
# stream = True returns an iterator, but stream = False would return a list
# stream = True case
for res in response: print(res, end='', flush=True)
client.stop_session()

For more information on sessions and batch processing, please take a look at our API Docs.