python-tokenizers: An implementation of today's most used tokenizers, with a focus on performance and versatility

An implementation of today's most used tokenizers, with a focus on performance and versatility.

⚠ This package has the following problems:

python-tokenizers(x86_64,devel-misc): python-tokenizers(x86_64,devel-misc) obsoletes python-tokenizers-py3 provided by python-tokenizers-py3(x86_64,base) which comes from same source name; ignoring
python-tokenizers(x86_64,devel-misc): aider-chat(noarch,base) requires python3.11dist(tokenizers) which is only provided by older package(s): python-tokenizers-py3(x86_64,base)
python-tokenizers(x86_64,devel-misc): python-litellm-py3(noarch,base) requires python3.11dist(tokenizers) which is only provided by older package(s): python-tokenizers-py3(x86_64,base)
need to be rebuilt: aider-chat(x86_64,base) python-litellm(x86_64,base)

Name:	python-tokenizers
Release:	0.23.1-2mamba
Group:	System/Libraries
Maintainer:	silvan
License:	Apache License 2.0
Size:	368.10 kB
Upstream URL:	https://github.com/huggingface/tokenizers
Build time:	2026-07-08 16:25:24

python-tokenizers.spec
tokenizers-0.23.1.tar.gz

	x86_64
Built RPMS	python-tokenizers python-tokenizers-debug
Build requirements	glibc-devel [1] libgcc [1] libpython314-devel [1] [2] libstdc++6-devel [1] python3[>=0:3.14.6-2mamba] [1] python3.14dist(huggingface-hub)[<2~~] [1] python3.14dist(huggingface-hub)[>=0.16.4)] [1]

Date	Packager	Release	Description
2026-07-08	Automatic Build System	0.23.1-2mamba	- rebuilt by autoport with build requirements: python3>=0:3.14.6-2mamba
2026-04-27	Automatic Build System	0.23.1-1mamba	- automatic version update by autodist
2026-01-20	silvan	0.22.2-1mamba	- package created using the webbuild interface

Build required by