CTAN has a new package: tex2tok

Jim Hefferon ftpmaint at alan.smcvt.edu
Thu Jun 30 21:43:27 CEST 2005

A new package has been installed at tug.ctan.org and should soon make its
way to your favorite mirror.

Thanks again,


The following information was provided by the package's contributor.

Name of contribution: tex2tok
Author's name: Jonathan Fine
Location on CTAN: /support/tex2tok
Summary description: tex2tok: convert a TeX source file into token
License type: gpl

Announcement text given by the package's contributor:
- - ----------------------------------------------------------------------
tex2tok: convert a TeX source file into tokens

$ tex '&tex2tok' '\LEX{story}'
This will create a file story.tok.
This file will contain one line for each token.
- - ----------------------------------------------------------------------

You can have a look at the package at
although you may get a better network connection by visiting a mirror 
of CTAN that is near to you; see 

More information about the Ctan-ann mailing list