FileHashIntel

tokenizers/implementations/bpe.py

6 distinct file contents recorded under this name ยท 60 PyPI records

Python source file Reference

A .py file contains Python source code in plain text. The Python interpreter reads the file and runs the code directly, without a separate compile step. Developers write scripts, modules, and full programs in this format. A text editor can open and change the file. Python is one of the most used programming languages, thus .py files are common on many systems.

A file extension is only a part of the file name. It does not give a guarantee about the format or the content of a file. But on Windows, the extension selects the program that opens the file.

Hashes for this file

2b59a832dea7a20b1eed2b42021330712e0f7129c8f5fa3cc179925853943e06 2528 bytes

  • tokenizers 0.2.0 tokenizers-0.2.0-cp35-cp35m-win_amd64.whl tokenizers-0.2.0-cp36-cp36m-win_amd64.whl tokenizers-0.2.0-cp37-cp37m-win_amd64.whl tokenizers-0.2.0-cp38-cp38-win_amd64.whl
  • tokenizers 0.2.1 tokenizers-0.2.1-cp35-cp35m-win_amd64.whl tokenizers-0.2.1-cp36-cp36m-win_amd64.whl tokenizers-0.2.1-cp37-cp37m-win_amd64.whl tokenizers-0.2.1-cp38-cp38-win_amd64.whl

8d83244e3e0500377a36a31db9864b485df5cfc9b0df5baed4c0612419dd4421 2462 bytes

  • tokenizers 0.2.0 tokenizers-0.2.0-cp35-cp35m-macosx_10_13_x86_64.whl tokenizers-0.2.0-cp35-cp35m-manylinux1_x86_64.whl tokenizers-0.2.0-cp36-cp36m-macosx_10_13_x86_64.whl tokenizers-0.2.0-cp36-cp36m-manylinux1_x86_64.whl tokenizers-0.2.0-cp37-cp37m-macosx_10_13_x86_64.whl tokenizers-0.2.0-cp37-cp37m-manylinux1_x86_64.whl tokenizers-0.2.0-cp38-cp38-macosx_10_13_x86_64.whl tokenizers-0.2.0-cp38-cp38-manylinux1_x86_64.whl
  • tokenizers 0.2.1 tokenizers-0.2.1-cp35-cp35m-macosx_10_13_x86_64.whl tokenizers-0.2.1-cp35-cp35m-manylinux1_x86_64.whl tokenizers-0.2.1-cp36-cp36m-macosx_10_13_x86_64.whl tokenizers-0.2.1-cp36-cp36m-manylinux1_x86_64.whl tokenizers-0.2.1-cp37-cp37m-macosx_10_13_x86_64.whl tokenizers-0.2.1-cp37-cp37m-manylinux1_x86_64.whl tokenizers-0.2.1-cp38-cp38-macosx_10_13_x86_64.whl tokenizers-0.2.1-cp38-cp38-manylinux1_x86_64.whl

a981817f600badeeeb2cc33e3c62a2c17a34374d9190b50353fa8afa745fdc81 2523 bytes

  • tokenizers 0.1.0 tokenizers-0.1.0-cp35-cp35m-win_amd64.whl tokenizers-0.1.0-cp36-cp36m-win_amd64.whl tokenizers-0.1.0-cp37-cp37m-win_amd64.whl tokenizers-0.1.0-cp38-cp38-win_amd64.whl
  • tokenizers 0.1.1 tokenizers-0.1.1-cp35-cp35m-win_amd64.whl tokenizers-0.1.1-cp36-cp36m-win_amd64.whl tokenizers-0.1.1-cp37-cp37m-win_amd64.whl tokenizers-0.1.1-cp38-cp38-win_amd64.whl

6a29e7d0247e2c2bbbb3a8cd0c80b180e64c0b3745e4c253293dcfc3c811adf6 2457 bytes

  • tokenizers 0.1.0 tokenizers-0.1.0-cp35-cp35m-macosx_10_13_x86_64.whl tokenizers-0.1.0-cp35-cp35m-manylinux1_x86_64.whl tokenizers-0.1.0-cp36-cp36m-macosx_10_13_x86_64.whl tokenizers-0.1.0-cp36-cp36m-manylinux1_x86_64.whl tokenizers-0.1.0-cp37-cp37m-macosx_10_13_x86_64.whl tokenizers-0.1.0-cp37-cp37m-manylinux1_x86_64.whl tokenizers-0.1.0-cp38-cp38-macosx_10_13_x86_64.whl tokenizers-0.1.0-cp38-cp38-manylinux1_x86_64.whl
  • tokenizers 0.1.1 tokenizers-0.1.1-cp35-cp35m-macosx_10_13_x86_64.whl tokenizers-0.1.1-cp35-cp35m-manylinux1_x86_64.whl tokenizers-0.1.1-cp36-cp36m-macosx_10_13_x86_64.whl tokenizers-0.1.1-cp36-cp36m-manylinux1_x86_64.whl tokenizers-0.1.1-cp37-cp37m-macosx_10_13_x86_64.whl tokenizers-0.1.1-cp37-cp37m-manylinux1_x86_64.whl tokenizers-0.1.1-cp38-cp38-macosx_10_13_x86_64.whl tokenizers-0.1.1-cp38-cp38-manylinux1_x86_64.whl

7d75eab9ef96de5c39cc086e59d0e7cfa5e221929f7bc5648d59d3853a5495c2 2439 bytes

  • tokenizers 0.0.13 tokenizers-0.0.13-cp35-cp35m-win_amd64.whl tokenizers-0.0.13-cp36-cp36m-win_amd64.whl tokenizers-0.0.13-cp37-cp37m-win_amd64.whl tokenizers-0.0.13-cp38-cp38-win_amd64.whl

ede120ed27c215473c77dcbd5a32bdd40c96bcf77fba5406ca308e64a6fb8c0d 2375 bytes

  • tokenizers 0.0.13 tokenizers-0.0.13-cp35-cp35m-macosx_10_13_x86_64.whl tokenizers-0.0.13-cp35-cp35m-manylinux1_x86_64.whl tokenizers-0.0.13-cp36-cp36m-macosx_10_13_x86_64.whl tokenizers-0.0.13-cp36-cp36m-manylinux1_x86_64.whl tokenizers-0.0.13-cp37-cp37m-macosx_10_13_x86_64.whl tokenizers-0.0.13-cp37-cp37m-manylinux1_x86_64.whl tokenizers-0.0.13-cp38-cp38-macosx_10_13_x86_64.whl tokenizers-0.0.13-cp38-cp38-manylinux1_x86_64.whl