hf-xet

Fast transfer of large files with the Hugging Face Hub.

35 个版本 Python >=3.8

安装

pip install hf-xet

poetry add hf-xet

pipenv install hf-xet

conda install hf-xet

描述

🤗 hf-xet - xet client tech, used in huggingface_hub

Welcome

hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.

Key features

♻ chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).

🤗 Python bindings: bindings for huggingface_hub package.

↔ network communications: concurrent communication to HF Hub Xet backend services (CAS).

🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.

Installation

Install the hf_xet package with pip:

pip install hf_xet

Quick Start

hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.

Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️

Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.

分类

Development Status :: 5 - Production/Stable License :: OSI Approved :: Apache Software License Programming Language :: Python :: 3 Programming Language :: Python :: 3 :: Only Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 Programming Language :: Python :: 3.13 Programming Language :: Python :: 3.14 Programming Language :: Python :: 3.8 Programming Language :: Python :: 3.9 Programming Language :: Python :: Free Threading Programming Language :: Python :: Free Threading :: 2 - Beta Programming Language :: Python :: Implementation :: CPython Programming Language :: Python :: Implementation :: PyPy Programming Language :: Rust Topic :: Scientific/Engineering :: Artificial Intelligence

版本列表

1.5.1 2026-06-08

1.5.0 2026-05-06

1.5.1rc1 2026-06-08

1.5.1rc0 2026-06-08

1.4.3 2026-03-31

1.4.2 2026-03-13

1.4.1 2026-03-12

1.4.0 2026-03-11

1.3.2 2026-02-27

1.3.1 2026-02-25

1.3.0 2026-02-24

1.2.1 2025-11-21

1.2.0 2025-10-24

1.1.10 2025-09-12

1.1.9 2025-08-27

1.1.8 2025-08-18

1.1.7 2025-08-06

1.1.6 2025-08-05

1.1.5 2025-06-20

1.1.4 2025-06-16

1.1.3 2025-06-04

1.1.2 2025-05-16

1.1.1 2025-05-12

1.1.0 2025-04-29

1.0.5 2025-04-25

1.0.4 2025-04-24

1.0.3 2025-04-09

1.0.2 2025-04-06

1.0.1 2025-04-06

1.0.0 2025-04-01

0.1.4 2025-03-05

0.1.3 2025-02-14

0.1.2 2025-01-15

0.1.1 2025-01-14

0.1.0 2025-01-10