python-tokenizers-py3: An implementation of today's most used tokenizers, with a focus on performance and versatility

An implementation of today's most used tokenizers, with a focus on performance and versatility

Nome:	python-tokenizers-py3
Versione:	0.22.2-1mamba
Architettura:	x86_64
Gruppo:	System/Libraries/Python
Dimensione:	2.63 MB
URL di origine:	https://github.com/huggingface/tokenizers
RPM sorgente:	python-tokenizers

Collegati	Fornisce	Rende obsoleti	Richiede	Raccomanda
python-tokenizers-debug	python-tokenizers-py3 = 0:0.22.2-1mamba python-tokenizers-py3(x86-64) = 0:0.22.2-1mamba python3.11dist(tokenizers) = 0:0.22.2 python3dist(tokenizers) = 0:0.22.2		ld-linux-x86-64.so.2()(64bit) [1] ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) [1] libc.so.6(GLIBC_2.34)(64bit) [1] libgcc_s.so.1()(64bit) [1] [2] libgcc_s.so.1(GCC_3.0)(64bit) [1] [2] libgcc_s.so.1(GCC_3.3)(64bit) [1] [2] libgcc_s.so.1(GCC_4.2.0)(64bit) [1] [2] libm.so.6()(64bit) [1] libm.so.6(GLIBC_2.29)(64bit) [1] libstdc++.so.6()(64bit) [1] libstdc++.so.6(CXXABI_1.3)(64bit) [1] libstdc++.so.6(CXXABI_1.3.8)(64bit) [1] libstdc++.so.6(GLIBCXX_3.4)(64bit) [1] python(abi) = 0:3.11 [1] [2] python3 >= 0:3.11 [1] python3 < 0:3.12 [1] python3.11dist(huggingface-hub) >= 0:0.16.4 [1] python3.11dist(huggingface-hub) < 0:2 [1] rtld(GNU_HASH) [1]

File forniti