Untitled

 avatar
unknown
python
a year ago
158 B
8
Indexable
llm = LLM(model="mistralai/Mistral-7B-Instruct-v0.3", 
            dtype="half", 
            tensor_parallel_size=4, 
            gpu_memory_utilization=0.2)
Editor is loading...
Leave a Comment