result

(base) jl@JLs-Laptop build % ./pytorch_inference
Current number of threads: 4
Model loaded successfully.
Batch Size: 1, Average Time: 1.86423e-05 seconds, Throughput: 53641.4 samples/second
Batch Size: 2, Average Time: 1.90694e-05 seconds, Throughput: 104880 samples/second
Batch Size: 4, Average Time: 2.47976e-05 seconds, Throughput: 161306 samples/second
Batch Size: 8, Average Time: 3.45569e-05 seconds, Throughput: 231502 samples/second
Batch Size: 16, Average Time: 6.15277e-05 seconds, Throughput: 260046 samples/second
Batch Size: 32, Average Time: 9.73409e-05 seconds, Throughput: 328742 samples/second
Batch Size: 64, Average Time: 0.000174497 seconds, Throughput: 366770 samples/second
Batch Size: 128, Average Time: 0.000330429 seconds, Throughput: 387375 samples/second
Batch Size: 256, Average Time: 0.000638286 seconds, Throughput: 401074 samples/second
Batch Size: 512, Average Time: 0.00125685 seconds, Throughput: 407367 samples/second
Batch Size: 1024, Average Time: 0.00267883 seconds, Throughput: 382256 samples/second