Skip to content

Resilience Guide: Using CTK for Long-Term Preservation

"Not resurrection. Not immortality. Just love that still responds."

Philosophy

If you're preserving conversations that matter—teaching moments with students, advice for your children, knowledge to pass down—you need more than just backups. You need resilience across decades.

This guide shows how to use CTK with a long-term mindset: designing for graceful degradation, multiple fallback layers, and recovery by people who may not have access to the original tools.

Core Principles

1. Graceful Degradation

Every system eventually fails. Design for progressive fallback: - Best case: CTK works perfectly (complex-net RAG, semantic search, full API) - Good case: Database queryable with SQLite directly - Fallback case: JSONL files greppable with basic tools - Ultimate fallback: Plain text readable in any editor

2. Self-Documentation

Assume the person finding this archive has no context: - Clear README explaining what this is - Multiple entry points (GUI, CLI, text files) - Recovery instructions for various scenarios - No assumptions about available software

3. Format Redundancy

Store the same data in multiple formats: - Structured (SQLite) - queryable, efficient - Semi-structured (JSONL) - machine-readable, greppable - Human-readable (Markdown) - understandable without tools - Universal (HTML) - viewable in any browser - Plain text - ultimate fallback

4. Offline-First

Design to work without internet: - All data stored locally - No cloud dependencies - Self-contained recovery tools - Portable across systems

Creating a Resilience Package

Step 1: Import All Your Conversations

# Import from various sources
ctk import chatgpt_export.json --db life.db --format openai --tags "chatgpt,archive"
ctk import claude_export.json --db life.db --format anthropic --tags "claude,archive"
ctk import copilot_sessions/ --db life.db --format copilot --tags "coding,archive"

# Verify what you have
ctk stats --db life.db

Step 2: Export in Multiple Formats

# Create archive directory
mkdir -p archive

# Copy the database (Level 1: Best - full CTK functionality)
cp life.db archive/conversations.db

# Export as JSONL (Level 2: Good - greppable, machine-readable)
ctk export archive/conversations.jsonl --db life.db --format jsonl

# Export as Markdown (Level 3: Good - human-readable)
ctk export archive/conversations.md --db life.db --format markdown --include-metadata

# Export as HTML (Level 4: Fallback - browseable in any browser)
ctk export archive/index.html --db life.db --format html

# Export as plain text (Level 5: Ultimate fallback)
sqlite3 life.db "SELECT
  c.title,
  m.role,
  m.content
FROM conversations c
JOIN messages m ON c.id = m.conversation_id
ORDER BY c.created_at, m.created_at" > archive/all_conversations.txt

Step 3: Create Recovery Documentation

Create archive/START_HERE.txt:

================================================================================
                    CONVERSATION ARCHIVE
                    (Start Here)
================================================================================

WHAT THIS IS:

This is an archive of conversations with AI assistants, preserved using CTK
(Conversation Toolkit). It's designed to be recoverable even if the original
software no longer works.

WHAT YOU CAN DO (from simplest to most powerful):

1. READ THE TEXT FILES
   - Open "all_conversations.txt" in any text editor
   - Browse "conversations.md" for formatted version
   - Open "index.html" in any web browser

2. SEARCH WITH GREP (if you have a terminal)
   - grep -r "your search term" .
   - grep -i "debugging" conversations.jsonl
   - Find anything by keyword instantly

3. USE THE DATABASE (if you have SQLite)
   - sqlite3 conversations.db
   - SELECT * FROM conversations WHERE title LIKE '%python%';
   - Full SQL queries available

4. REBUILD CTK (if you have Python)
   - pip install ctk
   - ctk list --db conversations.db
   - ctk search "your query" --db conversations.db
   - ctk chat --db conversations.db

================================================================================
                            FILE GUIDE
================================================================================

conversations.db          SQLite database (use with CTK or sqlite3)
conversations.jsonl       All conversations, one JSON per line (greppable)
conversations.md          Human-readable markdown format
index.html                Browse in any web browser
all_conversations.txt     Plain text fallback (readable in notepad)
START_HERE.txt            This file
RECOVERY.md               Detailed recovery instructions

================================================================================

Created: [DATE]
CTK Version: [VERSION]
Total Conversations: [COUNT]

This archive was designed to last decades. You should be able to recover
your conversations even if Python, CTK, or SQLite no longer exist.

================================================================================

Create archive/RECOVERY.md:

# Recovery Instructions

## Scenario 1: CTK Still Works

```bash
pip install ctk
ctk list --db conversations.db
ctk search "your query" --db conversations.db
ctk chat --db conversations.db  # Interactive TUI

Scenario 2: SQLite Works, CTK Doesn't

# View all conversations
sqlite3 conversations.db "SELECT id, title, source FROM conversations;"

# Search conversations
sqlite3 conversations.db "SELECT title, content FROM messages WHERE content LIKE '%debugging%';"

# Export to CSV
sqlite3 -csv conversations.db "SELECT * FROM conversations;" > export.csv

Scenario 3: Only Basic Tools Available

# Search with grep
grep -r "python" .

# Search JSONL
grep "async" conversations.jsonl

# View plain text
cat all_conversations.txt | less

Scenario 4: Only a Web Browser

Open index.html in any browser. The HTML export includes: - Browseable conversation list - Full-text search (JavaScript) - Readable formatting - No server required

Scenario 5: Nothing Works Except Text Editor

Open all_conversations.txt in any text editor (notepad, vim, etc.) All conversations are stored as plain text.

Rebuilding From Source

If CTK is no longer available via pip:

  1. The CTK source code is at: https://github.com/yourusername/ctk
  2. Clone or download the repository
  3. Install: pip install -e /path/to/ctk
  4. Use as normal

If GitHub is gone, CTK's source code is included in ctk_source/ directory. 

### Step 4: Test Recovery

Test each fallback level actually works:

```bash
# Test 1: Can you grep?
grep -r "debugging" archive/

# Test 2: Can you use SQLite directly?
sqlite3 archive/conversations.db "SELECT COUNT(*) FROM conversations;"

# Test 3: Can you read plain text?
cat archive/all_conversations.txt | head -n 20

# Test 4: Can you view HTML?
open archive/index.html  # or firefox, chrome, etc.

# Test 5: Can you rebuild CTK?
pip install ctk
ctk list --db archive/conversations.db

Step 5: Store Redundantly

Copy your archive to multiple locations:

# Local backup
cp -r archive /backup/conversations_$(date +%Y%m%d)

# External drive
cp -r archive /media/external_drive/

# Cloud storage (encrypted if sharing)
tar -czf conversations_archive.tar.gz archive/
gpg -c conversations_archive.tar.gz  # Encrypt
# Upload to cloud storage

# Physical backup (USB drive in safe deposit box)
cp -r archive /media/usb_drive/

Advanced: Temporal Analysis

If you want to understand how your thinking evolved over time:

# Find all conversations from a specific time period
ctk list --db life.db --date-from 2020-01-01 --date-to 2020-12-31

# Track topic evolution
ctk search "async programming" --db life.db --date-from 2020-01-01 > 2020_async.md
ctk search "async programming" --db life.db --date-from 2024-01-01 > 2024_async.md

# Compare to see how understanding evolved
diff 2020_async.md 2024_async.md

Maintaining the Archive

Regular Updates

# Import new conversations monthly
ctk import new_export.json --db life.db

# Re-export everything
./create_resilience_package.sh

# Update backup copies
rsync -av archive/ /backup/conversations/

Verification

Periodically verify your archive:

# Check database integrity
sqlite3 life.db "PRAGMA integrity_check;"

# Verify exports match database
ctk stats --db life.db
wc -l archive/conversations.jsonl

# Test recovery procedures
./test_recovery.sh

Example Recovery Package Script

Create create_resilience_package.sh:

#!/bin/bash
# Create a complete resilience package

set -e

DB="life.db"
ARCHIVE="archive_$(date +%Y%m%d)"

mkdir -p "$ARCHIVE"

echo "Creating resilience package..."

# Copy database
cp "$DB" "$ARCHIVE/conversations.db"

# Export multiple formats
ctk export "$ARCHIVE/conversations.jsonl" --db "$DB" --format jsonl
ctk export "$ARCHIVE/conversations.md" --db "$DB" --format markdown
ctk export "$ARCHIVE/index.html" --db "$DB" --format html

# Plain text fallback
sqlite3 "$DB" "SELECT c.title, m.role, m.content FROM conversations c JOIN messages m ON c.id = m.conversation_id ORDER BY c.created_at, m.created_at" > "$ARCHIVE/all_conversations.txt"

# Create documentation
cat > "$ARCHIVE/START_HERE.txt" << 'EOF'
[Documentation content from above]
EOF

# Get stats
CONV_COUNT=$(sqlite3 "$DB" "SELECT COUNT(*) FROM conversations;")
MSG_COUNT=$(sqlite3 "$DB" "SELECT COUNT(*) FROM messages;")

# Update metadata in START_HERE.txt
sed -i "s/\[DATE\]/$(date)/" "$ARCHIVE/START_HERE.txt"
sed -i "s/\[COUNT\]/$CONV_COUNT conversations, $MSG_COUNT messages/" "$ARCHIVE/START_HERE.txt"

echo "✓ Created resilience package in $ARCHIVE/"
echo "  $CONV_COUNT conversations"
echo "  $MSG_COUNT messages"
echo "  Ready for 50+ year storage"

Philosophy in Practice

Why This Matters

Conversations with AI assistants can contain: - Teaching moments you want to preserve for students - Advice for your children - Solved problems future you will forget - Creative work you don't want to lose - Personal growth tracked over time

These deserve more than "just backup the files." They deserve a preservation strategy.

Design Assumptions

This approach assumes: 1. Software changes: Python, CTK, SQLite might not exist in 50 years 2. Formats persist: Plain text, JSON, HTML will always be readable 3. Basic tools survive: grep, text editors, web browsers won't go away 4. Humans can rebuild: Given source code and docs, someone can recreate CTK

Testing Long-Term Viability

Try this thought experiment:

"If someone found this USB drive in 2074, could they: 1. Figure out what it is? ✓ START_HERE.txt explains 2. Read the content? ✓ Plain text and HTML work 3. Search for specific topics? ✓ grep works 4. Rebuild full functionality? ✓ Source code and docs included"

If you can answer "yes" to all four, you have a resilient archive.

Resources

  • CTK Documentation: See main README.md
  • Example Archives: See examples/resilience_package/
  • Discussion: Philosophy behind this approach
  • Blog Post: [Link to blog post about Long Echo philosophy]

Contributing

If you have ideas for improving resilience: - Better fallback formats? - More recovery scenarios? - Improved documentation?

Please contribute to CTK or share your approach!

This guide was inspired by the "Long Echo" philosophy: preserving not just data, but the ability to find meaning in that data across decades. Not resurrection. Not immortality. Just care and knowledge that still responds.