Untitled
unknown
python
a year ago
158 B
8
Indexable
llm = LLM(model="mistralai/Mistral-7B-Instruct-v0.3", dtype="half", tensor_parallel_size=4, gpu_memory_utilization=0.2)
Editor is loading...
Leave a Comment
llm = LLM(model="mistralai/Mistral-7B-Instruct-v0.3", dtype="half", tensor_parallel_size=4, gpu_memory_utilization=0.2)