Complete Guide

Everything you need to know about Miata Maestro

Contents

Overview Architecture Features Configuration AI Integration Debugging Development Security

Overview

Miata Maestro is a specialized web scraper designed to help Mazda Miata enthusiasts find NA Miatas (1989-1997) on Facebook Marketplace. Built with modern web technologies and AI integration, it provides intelligent analysis and automation for car hunting.

🎯 Purpose

Automate the tedious process of browsing Facebook Marketplace for specific Miata listings with intelligent filtering and AI-powered analysis.

🏗️ Design Philosophy

Modular architecture for easy debugging, zero configuration for quick setup, and privacy-first approach with session-only storage.

🚀 Performance

Processes 15-25 listings per minute with 2-5 second AI analysis times. Optimized for both speed and accuracy.

Architecture

Miata Maestro uses a clean, modular architecture with three core files, each handling a specific responsibility.

🖥️

main.js

320 lines

Responsibilities:

  • Express server setup and configuration
  • Session management and authentication
  • Route definitions and HTML templates
  • Real-time progress tracking via SSE
  • Coordinate calls to llm.js and scraper.js
Express.js Sessions SSE
🤖

llm.js

131 lines

Responsibilities:

  • Ollama API client and integration
  • Listing evaluation with structured prompts
  • Lowball message generation
  • Multi-provider LLM support
  • Error handling for AI services
Ollama llama3.1:8b Axios
🌐

scraper.js

580 lines

Responsibilities:

  • Puppeteer browser lifecycle management
  • Facebook login and navigation
  • Marketplace search and data extraction
  • Anti-detection and human simulation
  • Listing filtering and validation
Puppeteer Chrome DOM Parsing

Benefits of This Architecture

🐛 Easy Debugging: Issues are isolated to specific files based on their function
🔧 Modular Development: Work on features independently without affecting others
📈 Scalable: Easy to add new AI providers or scraping targets
🧪 Testable: Each module can be tested in isolation

Features Deep Dive

🔍 Smart Search & Filtering

Advanced filtering system designed specifically for NA Miata hunting:

Geographic Filtering:
  • ZIP code-based location targeting
  • Customizable search radius (1-500 miles)
  • Automatic coordinate conversion
Vehicle-Specific Filters:
  • Year range targeting (1989-1997 for NA)
  • Mileage thresholds with validation
  • Optional price limits
  • Transmission type detection
Content Filtering:
  • Automatic Miata keyword detection
  • Parts-only listing exclusion
  • Duplicate listing removal
  • Quality score assessment

🤖 AI-Powered Analysis

Local AI processing for intelligent listing evaluation:

📊 Market Analysis

  • Fair market value estimation
  • Price comparison with similar listings
  • Regional market trends

🔍 Condition Assessment

  • Pros and cons identification
  • Red flag detection
  • Maintenance history analysis

💬 Negotiation Support

  • Strategic lowball calculations
  • Natural message generation
  • Contextual negotiation angles

🎨 User Interface

Modern, responsive interface with glassmorphism design:

Visual Design: Clean glassmorphism aesthetic with smooth animations and hover effects
Real-time Updates: Server-sent events for live progress tracking during scraping
Responsive Layout: Works seamlessly on desktop, tablet, and mobile devices
Interactive Elements: One-click actions for evaluation, message generation, and copying

Configuration

Miata Maestro requires minimal configuration. Most settings are handled through the web interface.

🔧 Search Parameters

Parameter
Default
Description
zip
90210
ZIP code for search center
radius
50
Search radius in miles (1-500)
yearMin/Max
1990/1997
Year range for NA Miatas
maxMileage
100000
Maximum mileage threshold
maxPrice
null
Optional price limit
limit
20
Number of listings to process

🤖 AI Configuration

Ollama (Recommended)

Local AI processing with privacy and speed benefits.

Default Port: 11434
Model: llama3.1:8b
Temperature: 0.7 (evaluation), 0.8 (messages)

Custom LLM Endpoint

Use any OpenAI-compatible API service.

Format: http://localhost:8000/v1/complete
Authentication: Set via web interface
Timeout: 30 seconds

AI Integration

Miata Maestro's AI features provide intelligent analysis using local language models for privacy and speed.

🧠 How AI Analysis Works

1
Data Collection: Extract listing details including title, price, mileage, description, and images
2
Prompt Engineering: Structure data into specialized prompts for car evaluation
3
AI Processing: Send to local llama3.1:8b model via Ollama API
4
Result Parsing: Extract structured analysis including pros, cons, pricing, and recommendations

📝 Prompt Engineering

Carefully crafted prompts ensure consistent, useful analysis:

Evaluation Prompt Structure:

You are an expert Mazda Miata evaluator. Analyze this NA Miata listing:

LISTING DATA:
- Title: [listing title]
- Year: [year]
- Price: [price]
- Mileage: [mileage]
- Transmission: [transmission]
- Description: [description]

Provide analysis in this exact HTML format:

<strong>Pros:</strong>
• [positive aspects]

<strong>Concerns:</strong>
• [potential issues]

<strong>Accurate Price:</strong> $[market value]

<strong>Lowball:</strong> $[strategic offer]

💬 Message Generation

AI creates natural, contextual negotiation messages:

Casual Tone: Sounds like a real person texting, not an AI
Specific Issues: Mentions relevant concerns about the car
Strategic Pricing: Incorporates calculated lowball offers
Platform Appropriate: Optimized for Facebook Marketplace communication

Debugging & Development

Miata Maestro's modular architecture makes debugging straightforward. Issues are isolated to specific components.

🔍 Debug Mode

Enable debug mode for detailed logging and limited processing:

Web Interface: Click "🛑 End Early (Debug)" on search page
URL Parameter: Add ?debug=true to scrape-results URL
Effect: Processes only first 3 listings with enhanced logging

📊 Troubleshooting by Component

🖥️ Server Issues (main.js)

  • Port conflicts or binding errors
  • Session management problems
  • Route handling issues
  • HTML template errors

🤖 AI Problems (llm.js)

  • Ollama connection failures
  • Model loading issues
  • Prompt formatting errors
  • Response parsing problems

🌐 Scraping Issues (scraper.js)

  • Facebook login failures
  • Page navigation errors
  • Selector changes
  • Data extraction problems

🛠️ Development Tools

Screenshots: Auto-captured as debug-*.png on errors
Console Logs: Detailed progress logging with emojis for clarity
Error Messages: Structured error responses with troubleshooting hints
Network Monitoring: Optional request/response logging for debugging

Development

Contributing to Miata Maestro or customizing it for your needs.

🏗️ Project Structure

Miata Maestro/
├── main.js              # Express server & routes
├── llm.js               # AI evaluation logic  
├── scraper.js           # Puppeteer automation
├── package.json         # Dependencies
├── public/
│   └── styles.css       # Glassmorphism UI
├── fb_user_data/        # Browser session storage
├── docs/                # Documentation website
└── debug-*.png          # Debug screenshots

🔧 Development Setup

1. Fork Repository: Create your own copy on GitHub
2. Install Dependencies: npm install --save-dev nodemon
3. Development Mode: npx nodemon main.js for auto-reload
4. Test Changes: Use debug mode for faster iteration

🎯 Extension Ideas

🚗 Other Car Models

Adapt filtering logic for different vehicles (S2000, 350Z, etc.)

📊 Analytics Dashboard

Track price trends, listing frequency, and market insights

🔔 Notification System

Email/SMS alerts for listings matching specific criteria

🌍 Multiple Platforms

Extend to Craigslist, AutoTrader, or other marketplaces

Security & Privacy

Miata Maestro prioritizes user privacy and security with session-only storage and local AI processing.

🔒 Privacy Features

Session-Only Storage: Facebook credentials are never permanently stored
Local AI Processing: All analysis happens on your machine, not in the cloud
No Data Collection: No analytics, tracking, or user data collection
Open Source: Full transparency with GNU GPLv3 license

⚡ Security Best Practices

Credential Handling:
  • Credentials stored only in server memory
  • Automatic cleanup on session end
  • No logging of sensitive information
Network Security:
  • HTTPS-only connections to Facebook
  • Secure cookie configuration
  • Rate limiting to prevent abuse
Browser Automation:
  • Human-like behavior simulation
  • Random delays between actions
  • Respectful scraping practices

⚠️ Important Notes

  • Facebook Terms: Use responsibly and in accordance with Facebook's terms of service
  • 2FA: Temporarily disable two-factor authentication if login fails
  • Account Safety: Consider using a secondary Facebook account for scraping
  • Rate Limits: Don't run multiple instances simultaneously