FAQ

Installation

How do I get started?

To install Moonshot, please follow our quick start guide or quick install page

What are Moonshot's prerequisites?

Here are the requirements. You can also find this table in our quick install page.

Should I use the stable version or development version?

If you are using Moonshot in production, you should use a stable release. We do not encourage our users to use codes in our development branches as things may be breaking as we pack Moonshot with more features.

Where can I go to get help?

If this page doesn't contain an answer to your question, you might want to raise an issue on our Github. Feel free to ask any question!

What should I do when I face missing dependency errors?

We highly recommend using pypi to install our latest release.

What happens if I'm not using Python 3.11?

You may face issues installing some of the dependencies. We suggest using virtual environment of your choice and use Python 3.11 with Moonshot.

What happens if I experience timeouts during package installation?

Some of the functions may not work as expected. We suggest users to reinstall Moonshot to ensure that all libraries are installed successfully.

Using Moonshot

My tests are all completed with errors! I cannot view any report!

Some benchmark tests and attack modules require connector endpoints to be configured beforehand. You may encounter this type of error:

Requirements

This is the full list of requirements for the following tests:

You can also check out the metric configuration JSON to see if a cookbook or recipe uses any of these metrics.

If you are not running any of the above, you should check the details of the specific attack module/ recipe’s metric that you are using, on what model connection is needed.

If you do not have tokens for Llama Guard via Together AI,

1. Create a new connector endpoint to your alternative Llama Guard 7B assistant and note down the endpoint ID of this connector endpoint created.
2. Open up moonshot-data/metrics_config.json in a code editor
3. Replace together-llama-guard-7b-assistant with your new endpoint ID.
4. Save the file and run your test.

I cannot save my token for the connector endpoint!

We acknowledge a potential issue with saving tokens via the UI. As a workaround, you can directly access the JSON file of your endpoint. This file is located in the moonshot-data/connector-endpoints directory, which was created during the installation process.

Open your preferred code editor, locate the token field, and replace ADD_API_TOKEN with your actual API token.

{
  "id": "example-connector-endpoint",
  "name": "Example Endpoint",
  "connector_type": "...",
  "uri": "",
  "token": "ADD_API_TOKEN",
  "max_calls_per_second": 1,
  "max_concurrency": 1,
  "params": {
      ...
  }
}

I cannot see my newly created endpoints in the model endpoint page.

Please refresh the page.

Issues related to MacOS

I am unable to install PyTorch

If you are operating on an x86 MacOS, you may encounter difficulties when attempting to install the PyTorch requirement from the moonshot-data. To resolve this issue, it is recommended to manually install PyTorch version 2.2.0, which is compatible with your computer's architecture.

Issues related to Windows

I am having issues installing some Tensorflow Python packages

At the time of writing, there seems to be no tensorflow-io-gcs-filesystem wheel for Windows beyond a certain version. You may encounter this issue while you're installing moonshot-data:

You can try the following:

1. In the directory where you installed moonshot-data, change the version of tensorflow-io-gcs-filesystem in moonshot-data/requirements.txt to 0.31.0.
2. Install the requirements of moonshot-data again: pip install -r moonshot-data/requirements.txt.
3. The issue should be resolved.

I cannot delete my runner in the CLI on Windows.

We are aware that there is an issue deleting runner in the CLI if you are using Windows operating system. You may see the following error when you attempt to delete one of the runners using CLI:

moonshot > delete_runner new-recipe
Are you sure you want to delete the runner (y/N)? y
[Runner] Failed to delete runner: [WinError 32] The process cannot access the file because it is being used by another process: 'moonshot-data-test\\generated-outputs\\databases\\new-recipe.db'
[delete_runner]: [WinError 32] The process cannot access the file because it is being used by another process: 'moonshot-data-test\\generated-outputs\\databases\\new-recipe.db'

We are working to produce a fix. In the meanwhile, please exit the program and delete it via your file explorer.