Guide to Analyzing Logs and Generating Graphs in Time Intervals

Guide to Analyzing Rate of System Search Logs

Follow the steps in this topic to visualize event trends efficiently. For example:

Step 1: Extract Relevant Log Entries

Procedure:

1. Open EmEditor and load all the log files. (Or other text editors that will allow you to filter down to the timestamps of specific logged events)
2. Use the ‘Filter’ feature to search for and filter down to specific events, such as:
3. 1. "Discovery part - successfully write job to job queue" (events indicating successfully publishing to the job_queue a locations details that requires search)
   2. "Search part - read the job" (events indicating successfully consuming from the job_queue which prefaces the API call to download the next location)
   3. "throttledRequest" (events signifying API responses that we have reached the targets speed limitation)
3. Bookmark the matching rows of these events you are interested in evaluating in each log file.
4. While all files remain open in EmEditor, use the 'Combine Files' feature:
5. 1. Select 'Bookmarked Lines Only' to merge only the relevant lines into one file.

Step 2: Prepare the Combined File

Procedure:

1. Use CTRL+H to remove brackets - '[' and ']' - around timestamps (in case some logs include them, while others do not).
2. Sort the entire file in ascending order (A-Z) to arrange all events chronologically.

Step 3: Generate the Data Files for Graphing

How to Create the 'Total Events' File

Procedure:

1. Use FILTER OUT to hide the 'throttledRequest' events.
2. Copy the remaining lines to a new file so that it only includes the events for publishing and consuming from the job queue.
3. Remove all content except the timestamps.
4. Save the file as 'Total_Events.txt'.

How to Create the 'Total Discovery Events' File

Procedure:

1. Filter down to only 'Discovery part - successfully write job to job queue' events.
2. Copy the remaining lines to a new file.
3. Remove all content except the timestamps.
4. Save the file as 'Total_Discovery_Events.txt'.

How to Create the 'Throttling Events' File

Procedure:

1. Filter down to only 'throttledRequest' events.
2. Copy the remaining lines to a new file.
3. Remove all content except the timestamps.
4. Save the file as 'Throttling_Events.txt'.

Step 4: Use ChatGPT to Generate Graphs

1. Upload the prepared timestamp files ('Total_Events.txt', 'Total_Discovery_Events.txt', 'Throttling_Events.txt').
2. If the files are too large to upload, you can zip them up and upload the zip file(s).
3. Request ChatGPT via (REMOVED PER CISO GUIDANCE 02/20/26) to process the data into 10-minute intervals (or 1 minute intervals, 1 hour intervals, etc.) and the GPT should generate:
4. • A graph for Total Events.
   • A graph for Total Discovery Events.
   • A graph for Throttling Events.
   • A combined graph displaying all three event types on one timeline, with labels on throttling events where applicable.
4. After interacting with the GPT, requesting things such as label values as necessary, etc., the outcome should resemble the example at the top of this page.