Help 1D Data Explorer

Welcome to the 1D Data Explorer -- a demonstration tool originally created by Mikhael Margolin and me to discover patterns and synonym-like relationships in one-dimensional data streams. The tool is based on a simple algorithm that can be expressed in just a few lines of code.

More advanced versions of this algorithm have since been successfully applied across various domains, including marketing, finance, stock trading strategies, and behavioral analytics.

Follow the steps below to explore the tool using several preloaded sample datasets. If you'd like to go further, you're welcome to modify the code and adapt it to your own use case -- full access is provided as here is nothing beyond the standard JavaScript.

Getting Started

• Select Atom: An atom is an indivisible unit of data. You choose the atom type -- for example, individual characters ('Char'), words ('Word'), or any other elementary pieces of information you define yourself. Any type of atom can be used to discover patterns in sequential data.


• Select Data: Your input should be a delimited sequence of atoms. Each atom and its delimiter are represented as textual symbols. These can be actual human-readable items (e.g., {atom = char, del = null} or {atom = word, del = space}) or symbolic placeholders (e.g., {atom = webpage URL, del = any string}).


• Select Defaults: The algorithm includes several tunable parameters. To begin, we recommend using the default settings without diving into the technical details. You can always experiment with these options later.


• Run First Chunk: Data is processed in chunks. You can select the chunk size using the first dropdown menu below. The default chunk size is one million atoms. If your atom type is 'char', processing that chunk may take less than 10 seconds. For larger atoms (e.g., full words, phrases, or long keys like URLs), it may take longer. In that case, we recommend using a smaller chunk size. When ready, click 'Run First Chunk' and wait for the analysis results.


• View Results: Use the tabs in the main window to explore the analysis output.

Key Features

Patterns Tab

Displays identified patterns in a table with the following columns:

• Pattern: A recurring sequence or structure in your data.
• Frequency: The number of times the pattern appears.
• Length: The average size of the pattern (value divided by frequency).
• Value: The total value or weight of the pattern.
• ID: A row index or reference identifier.

Synonyms Tab

Displays contextually similar elements in a table with the following columns:

• Pattern: A primary data element.
• Co-Pattern: A related element that appears in similar contexts.
• Importance: Strength of the similarity or co-occurrence relationship.
• ID: Row index or reference to the related pattern.

Dynamics Tab

Shows how the algorithm learns and adapts over time, tracking the evolution of discovered patterns.

Outcomes Tab

Summarizes the statistical results and metrics of the current pattern discovery session.