CTAN has a new package: tex2tok

[Jim Hefferon]

A new package has been installed at tug.ctan.org and should soon make its
way to your favorite mirror.

Thanks again,
Jim

======================================================================

The following information was provided by the package's contributor.

Name of contribution: tex2tok
Author's name: Jonathan Fine
Location on CTAN: /support/tex2tok
Summary description: tex2tok: convert a TeX source file into token
License type: gpl

Announcement text given by the package's contributor:
- - ----------------------------------------------------------------------
tex2tok: convert a TeX source file into tokens

usage:  
$ tex '&tex2tok' '\LEX{story}'
This will create a file story.tok.
This file will contain one line for each token.
- - ----------------------------------------------------------------------

You can have a look at the package at
  http://tug.ctan.org/tex-archive/support/tex2tok
although you may get a better network connection by visiting a mirror 
of CTAN that is near to you; see 
  http://www.ctan.org/tex-archive/README.mirrors
.