unillm package

Submodules

unillm.unillm module

class unillm.unillm.ChatGPT(api_key=None, model_id='gpt-3.5-turbo')

Bases: UniLLMBase

Class representing a ChatGPT model for generating text responses.

Initializes the ChatGPT model with an API key and model name.

Args:

api_key (str, optional): The API key for the OpenAI service. If not provided, it’s fetched using get_api_key.

model_name (str, optional): The name of the GPT model to use. Default is gpt-3.5-turbo.

generate_response(message)

Generate a text response using the ChatGPT model.

Args:

message (str): The input message to respond to.

Returns:

str: The generated response.

class unillm.unillm.Claude(api_key=None, model_id='claude-3-opus-20240229')

Bases: UniLLMBase

Class representing a Claude model for generating text responses.

Initializes the Claude model with an API key and model ID.

Args:

api_key (str, optional): The API key for Claude’s services. If not provided, it’s fetched using get_api_key.

model_id (str): Identifier for the Claude model. Default is claude-3-opus-20240229

generate_response(message, max_tokens=1000, temperature=0)

Generate a text response using the Claude model.

Args:

message (str): The input message to respond to.

max_tokens (int): Maximum number of tokens to generate.

temperature (float): Temperature for generation.

Returns:

str: The generated response.

class unillm.unillm.CommandRPlus(model_id='CohereForAI/c4ai-command-r-plus-4bit')

Bases: UniLLMBase

Class representing the CommandRPlus model for generating text responses.

Initializes the CommandRPlus model with specified parameters.

Args:

model_id (str): Identifier for the model. Default is “CohereForAI/c4ai-command-r-plus-4bit”.

generate_response(user_input)

Generate a text response using the CommandRPlus model.

Args:

user_input (str): User’s input text to be incorporated into the message template.

Returns:

str: The generated text output, cleaned of any system tokens or command prompts.

class unillm.unillm.Llama2(model_id='meta-llama/Llama-2-7b-chat-hf', peft_path=None, max_new_tokens=50, top_p=0.95, top_k=50, temperature=1.0)

Bases: UniLLMBase

Class representing a Llama model for generating text responses.

Initializes the Llama model with specified parameters.

Args:

model_id (str): Identifier for the Llama model. Default is meta-llama/Llama-2-7b-chat-hf.

peft_path (str, optional): Path to the PEFT model.

max_new_tokens (int): Maximum new tokens to generate.

top_p (float): Nucleus sampling probability.

top_k (int): Top k filtering.

temperature (float): Temperature for generation.

generate_response(message)

Generate a text response using the Llama model.

Args:

message (str): The input message to respond to.

Returns:

str: The generated response.

get_prompt(message, chat_history=[], system_prompt='')

Constructs the prompt for the Llama model from the given message and chat history.

Args:

message (str): The input message.

chat_history (list): A list of tuples containing the chat history.

system_prompt (str): A prompt describing the system’s role.

Returns:

str: The constructed prompt.

class unillm.unillm.Llama3(model_id='meta-llama/Meta-Llama-3-8B-Instruct', device='cuda', max_new_tokens=256, temperature=0.6, top_p=0.9)

Bases: UniLLMBase

Class representing the Llama3 model for generating text responses.

Initializes the Llama3 model with specified parameters.

Args:

model_id (str): Identifier for the Llama model. Default is meta-llama/Meta-Llama-3-8B-Instruct. device (str): Device to run the model on, default is ‘cuda’. max_new_tokens (int): Maximum number of new tokens to generate. temperature (float): Temperature parameter for sampling. top_p (float): Top p parameter for nucleus sampling. torch_dtype (torch.dtype): Torch data type for the model, default is bfloat16.

generate_response(user_input)

Generate a text response using the Llama3 model.

Args:

user_input (str): User’s input text to be incorporated into the message template.

Returns:

str: The generated text output.

class unillm.unillm.Mistral(model_id='mistralai/Mistral-7B-Instruct-v0.2')

Bases: UniLLMBase

Class representing a Mistral model for generating text responses.

Initializes the Mistral model with a specified model ID.

Args:

model_id (str): Identifier for the Mistral model. Default is mistralai/Mistral-7B-Instruct-v0.2.

generate_response(message, max_new_tokens=1024, temperature=0.6, top_p=0.9, top_k=50, repetition_penalty=1.2)

Generate a text response using the Mistral model.

Args:

message (str): The input message to respond to.

max_new_tokens (int): Maximum new tokens to generate.

temperature (float): Temperature for generation.

top_p (float): Nucleus sampling probability.

top_k (int): Top k filtering.

repetition_penalty (float): Repetition penalty.

Returns:

str: The generated response.

class unillm.unillm.MistralAI(api_key=None, model_id='mistral-large-latest')

Bases: UniLLMBase

Class representing a MistralAI model for generating text responses via API.

Initializes the MistralAI model with an API key and model identifier.

Args:

api_key (str, optional): The API key for MistralAI’s services. If not provided, it’s fetched using get_api_key.

model_id (str): The model identifier for MistralAI. Default is mistral-large-latest.

generate_response(message)

Generate a text response using the MistralAI model.

Args:

message (str): The input message to respond to.

Returns:

str: The generated response.

class unillm.unillm.RAG(data_path, api_key=None, ragmodel=None)

Bases: UniLLMBase

Class representing a Retrieval-Augmented Generation (RAG) model for generating text responses.

Initializes the RAG model with the data path, API key, and model identifier.

Args:

data_path (str): The path to the data for the RAG model.

api_key (str, optional): The API key for the service. If not provided, it’s fetched using get_api_key.

ragmodel (str, optional): Identifier for a specific RAG model. e.g. meta-llama/Llama-2-7b-chat-hf. If ragmodel is not set, it will use chatgpt by default.

generate_response(message)

Generate a text response using the RAG model.

Args:

message (str): The input message to respond to.

Returns:

str: The generated response.

class unillm.unillm.UniLLM(model_type, **kwargs)

Bases: object

A factory class to initialize and interact with various UniLLM models.

Initializes the specified UniLLM model type with the given arguments.

Args:

model_type (str): The type of model to initialize.

**kwargs: Additional keyword arguments to pass to the model’s constructor.

generate_response(message)

Generates a response using the initialized model.

Args:

message (str): The message to generate a response for.

Returns:

str: The generated response.

initialize_model(model_type, **kwargs)

Initializes a specific model based on the model type.

Args:

model_type (str): The type of model to initialize. **kwargs: Additional keyword arguments for model initialization.

Returns:

UniLLMBase: An instance of the specified model type.

Raises:

ValueError: If the specified model type is unsupported.

class unillm.unillm.UniLLMBase

Bases: object

Base class for all UniLLM models providing a template for generating responses.

generate_response(message)

Abstract method to generate a response to a given message.

Args:

message (str): The message to respond to.

Raises:

NotImplementedError: If the subclass does not implement this method.

unillm.unillm.cmd(model_type=None, **kwargs)

Command line interface to interact with various UniLLM models.

Args:

model_type (str, optional): The type of model to interact with. If not specified, the user will be prompted to choose one.

**kwargs: Additional keyword arguments to pass to the model’s constructor.

unillm.unillm.get_all_subclasses(cls)

unillm.unillm.get_api_key(model_type)

Retrieve the API key for a given model type from a YAML configuration file.

Args:

model_type (str): The type of the model for which the API key is requested.

Returns:

str: The API key for the requested model type.

unillm.unillm.run_cmd()

Initializes and runs the command line interface using the Fire library.

unillm.version module

Module contents