Common methods collection used by harmonyos drivers.

TLLM_QMM strips the implementation of quantized kernels of Nvidia's TensorRT-LLM, removing NVInfer dependency and exposes ease of use Pytorch module. We modified the dequantation and weight preprocessing to align with popular quantization alogirthms such as AWQ and GPTQ, and combine them with new FP8 quantization.

A highly optimized LLM inference acceleration engine for Llama and its variants.

知乎 x RxJava Meetup

Elastic cache solution on Kubernetes