Untitled

mail@pastecode.io avatar
unknown
python
15 days ago
158 B
5
Indexable
Never
llm = LLM(model="mistralai/Mistral-7B-Instruct-v0.3", 
            dtype="half", 
            tensor_parallel_size=4, 
            gpu_memory_utilization=0.2)
Leave a Comment