CBF, Doc2Vec and overlapping chunks

Introduction

One important aspect of training models of fiction texts, and which we have disregarded so far, is the matter for overlapping chunks. If you, as we have done so far, cut a text into arbitrary chunks based on number of words, then you may lose important "connections" between words at the edges of the chunks, i.e. you may cut a sentence or paragraph into two at a point (edge) where a train of thought, a plot or a (important, character) description is being developed in the text. To remedy this, you are advised to have an overlap strategy to help "maintain semantic continuity between chunks" (Copilot Chat). Copilot Chat goes on to recommend:

Empirical Recommendations
For fiction texts, which often rely heavily on narrative flow and context, overlaps help preserve coherence. Based on research and practice:

• Overlap of 50–100 words is often effective for chunk sizes of 200–300 words.
• Overlap of 25–50% of the chunk size is a good starting point.

This would entail that I, with my chunk sizes of 100 and 1,000 words, should have overlaps of between 25-50 and 250-500 words respectively.

ChatGPT-5 gives these recommendations:

Short answer
There isn’t a single “optimal” overlap for all fiction corpora, but a robust rule of thumb for Doc2Vec on chunked fiction is:

• Overlap ≈ max (2 × window_size, 20–40% of chunk length), capped around 150–200 words.
• For common settings (chunk length 200–500 words, window 5–10), this usually means 50–150 words of overlap (roughly 25–33%).

Here are some further recommendations from ChatGPT-5:

If you need a starting point for fiction:

• Chunk length L = 300-400 words
• Window w = 8
• Overlap 100 words (≈ 25–33%)
• Sentence-aware splitting with overlaps aligned on sentence boundaries
• Use PV-DM (or PV-DM + PV-DBOW concatenation) with negative sampling and subsampling

In what follows, we shall test (some of) these recommendations to see if they have profound impact on our cosine similarity scores. I will not try to use PV-DM or go into the PV-DM or PV-DBOW concatenation suggestion.

Results

I tested two models trained on different inputs: In 1), I used a chunk size of 100 lemmas with a 20-lemma overlap, and in 2) I chose 500 lemmas and an overlap of 100 lemmas. In  the latter case, I also extended the window to 8 lemmas, as opposed to the/my "normal" 5-lemma window. 

Increasing chunk size, with a large overlap and an expanded window resulted in a much lower SC score for the top two chunks (0.53) compared to using 100-lemma chunks both with and without overlap. More interestingly (perhaps), was that the top two chunks changed when introducing overlap. Part of one of the chunks that came up top in our last experiment (bold, italic) (link) was paired with a different chunk this time and got a slightly lower score.

All in all, it does not seem to me that using overlaps makes a huge different for my model and the cosine similarity comparisons.

As an afterthought, I decided to try out training a model with overlaps using words, since I am still unsure about my decision to change proper nouns to tags, e.g. propn-Z1, when training the model using lemmas. The chunk size was again 100 words, but with an overlap of 10 words. The top pair using words was as high as 0.79, which surprised me bit. Can it be that using words creates a better model than using lemmas if testing cosine similarities are the goal? The pair that got this high score where these:

Again, the same chunk comes up for MicDib1, but it is paired with a different one from MicDib2. When I asked Copilot Chat why a language model would rate these two chunks as quite similar and giving them high SC score, the AI bot pointed to these factors: 

1. Narrative Style and Genre

Both paragraphs are written in a fictional narrative style—they describe scenes with a focus on character observation, setting, and subtle tension.

2. Descriptive Language

Both use rich, sensory detail:

• “sharply triangular as the jib of a sailing boat” and “grey with glints of blue” evoke visual imagery.
• “black brogue shoe resting on its side” and “polish badly scuffed” do the same.

3. Character Focus and Emotional Distance

Both paragraphs center on characters being observed or doing the observing, with a tone of detachment or aloofness:

• The stranger is described as “cynical, suave and aloof.”
• Zen is methodical, quiet, and focused on a clue.

The ChatBot calculated the CS score based on the two chunks and gave them a score of 0.38. The conclusion of this experiment of introducing overlap between chunks seems to be that it does not matter that much for our data and Doc2Vec model when the goal is to calculate cosine similarity between chunks of texts that previously have achieved a high similarity score.

As a final nod to the two texts I have dissected in this and the earlier post, let me compare the topics deduced from them using a Non-negative Matrix Factorisation topic-modelling approach.

The first topics for both texts is almost identical. But then, there is no surprise that detective fiction revolves around personal names (propn_z1), religion and the supernatural (propn_s9), geographical names (propn-z2) and time (propn_t1). Apart from think and tell the list of words for topic 1 match. However, looking at the next nine topics it is more difficult to find overlapping words. The exceptions are consider, moment, perfect, railway, simply and totally, but they do not give a clear indication that the two texts are detective novel written by the same author within a four-year period and are about the same detective hero. Copilot Chat does, however, point to the two texts having similar thematic clusters:

1. Emotions or judgments: admire, appal, persuade, suspect, appreciate
2. Communication or information: document, informant, directory, discuss
3. Places or objects: hospital, garden, helicopter, pizza, notebook