Version 1 uses direct text processing, which leads to the same KList lookups occurring over and over - completely inefficiently. It was my goal for the code to be true to an intuition about the "moving topic", so pre-processing the text wasn't allowed. Now for performance I may need to move past intuition and need to conceptualize entities that are not directly intuitive. In particular concepts of 'token' need to be replaced with concepts of filled VAR. A sequence of tokens is replaced by a sequence of filled VARs called a "segment". Tokenizing is replaced by segmentation. All other operations, up to the NWReader, need to be re-written using segmented text.
But I have to develop new intuitions for segments. They will be the "spatial curves" to which we fit the moving frames of narrative.
I like that last sentence.
No comments:
Post a Comment