Skip to main content

FireflyChat

FireflyChat is a graphical application platform for LLM developed by the Firefly team. Users can quickly experience the actual use of various large models on the visual WebUI provided by FireflyChat.

_images/fireflychat.png

Installation​

You need to obtain the required installation package from the Download Center to install FireflyChat. There are two installation packages, which are fireflychat*.deb and libfirefly-ai*.deb.

After obtaining the installation package, transfer it to AIBOX-1684X and run the following command to install it:

# Install venv
sudo apt update
sudo apt install python3-venv

# Install FireflyChat
sudo dpkg -i libfirefly-ai*.deb
sudo dpkg -i fireflychat*.deb

After installing FireflyChat, you need to import the model file to FireflyChat to use it properly.

Model import​

LLM​

Users can obtain the model files (.bmodel) provided by the Firefly team from the LLM model resource in the Download Center.

After obtaining the model file, transfer it to AIBOX-1684X and run the following command to import it:

# Move the model file location, where <model_path> replace with the actual model file path
sudo mv <model_path> /models/LLM/

Embedding model​

Users can obtain the model files (.bmodel) provided by the Firefly team from the embedding model resource in the Download Center.

After obtaining the model file, transfer it to AIBOX-1684X and run the following command to import it:

# Move the model file location, where <model_path> replace with the actual model file path
sudo mv <model_path> /models/embedding/

Restart FireflyChat​

After adding or modifying the required model files, you need to restart FireflyChat for the new model to take effect:

sudo systemctl restart FireflyChat

Software usage​

WebUI usage (Main usage methods)​

FireflyChat will run automatically after it is installed successfully. You can directly access FireflyChat by entering <ip_address>:7860 in the browser.

Note: <ip_address> must be replaced with the actual IP address, for example: 172.16.11.18:7860.

Software background management​

FireflyChat is an application registered as the systemd service. You can control the running status of FireflyChat by running the following commands:

# Stop FireflyChat
sudo systemctl stop FireflyChat

# Restart FireflyChat
sudo systemctl restart FireflyChat

# Disable the automatic startup of FireflyChat
sudo systemctl disable FireflyChat

# Viewing run logs
sudo journalctl -u FireflyChat