project-development – Agents & Automation skill for AI agents

Project Development Methodology

This skill covers the principles for identifying tasks suited to LLM processing, designing effective project architectures, and iterating rapidly using agent-assisted development. The methodology applies whether building a batch processing pipeline, a multi-agent research system, or an interactive agent application.

When to Use

Activate this skill when:

• Starting a new project that might benefit from LLM processing
• Evaluating whether a task is well-suited for agents versus traditional code
• Designing the architecture for an LLM-powered application
• Planning a batch processing pipeline with structured outputs
• Choosing between single-agent and multi-agent approaches
• Estimating costs and timelines for LLM-heavy projects

Core Concepts

Task-Model Fit Recognition

Not every problem benefits from LLM processing. The first step in any project is evaluating whether the task characteristics align with LLM strengths. This evaluation should happen before writing any code.

LLM-suited tasks share these characteristics:

| Characteristic | Why It Fits |

|----------------|-------------|

| Synthesis across sources | LLMs excel at combining information from multiple inputs |

| Subjective judgment with rubrics | LLMs handle grading, evaluation, and classification with criteria |

| Natural language output | When the goal is human-readable text, not structured data |

| Error tolerance | Individual failures do not break the overall system |

| Batch processing | No conversational state required between items |

| Domain knowledge in training | The model already has relevant context |

LLM-unsuited tasks share these characteristics:

| Characteristic | Why It Fails |

|----------------|--------------|

| Precise computation | Math, counting, and exact algorithms are unreliable |

| Real-time requirements | LLM latency is too high for sub-second responses |

| Perfect accuracy requirements | Hallucination risk makes 100% accuracy impossible |

| Proprietary data dependence | The model lacks necessary context |

| Sequential dependencies | Each step depends heavily on the previous result |

| Deterministic output requirements | Same input must produce identical output |

The evaluation should happen through manual prototyping: take one representative example and test it directly with the target model before building any automation.

The Manual Prototype Step

Before investing in automation, validate task-model fit with a manual test. Use an authorized synthetic or minimized input in an approved model interface; do not upload proprietary data to an unapproved service. Evaluate the output quality. This takes minutes and prevents hours of wasted development.

This validation answers critical questions:

• Does the model have the knowledge required for this task?
• Can the model produce output in the format you need?
• What level of quality should you expect at scale?
• Are there obvious failure modes to address?

A failed prototype is a diagnosis point, not proof that every architecture fails. A successful example is a starting point, not scale validation; test representative and adversarial cases with held-out evaluation.

Pipeline Architecture

LLM projects benefit from staged pipeline architectures where each stage is:

• Discrete: Clear boundaries between stages
• Idempotent: Repeating an operation does not duplicate its externally visible effects; identical LLM text is a separate determinism property
• Cacheable: Intermediate results persist to disk
• Independent: Each stage can run separately

The canonical pipeline structure:

acquire → prepare → process → parse → render

1. Acquire: Fetch raw data from sources (APIs, files, databases)

2. Prepare: Transform data into prompt format

3. Process: Execute LLM calls (the expensive, non-deterministic step)

4. Parse: Extract structured data from LLM outputs

5. Render: Generate final outputs (reports, files, visualizations)

Acquisition can change over time; preparation, parsing and rendering can be deterministic only with pinned inputs, versions and settings. Stage 3 is non-deterministic and expensive. This separation allows re-running the expensive LLM stage only when necessary, while iterating quickly on parsing and rendering.

File System as State Machine

Use the file system to track pipeline state rather than databases or in-memory structures. Each processing unit gets a directory. Each stage completion is marked by file existence.

data/{id}/
├── raw.json         # acquire stage complete
├── prompt.md        # prepare stage complete
├── response.md      # process stage complete
├── parsed.json      # parse stage complete

File existence alone does not establish completion or idempotency. Write temporary files and atomically rename only after validation; record input hashes, model/prompt/tool versions, schema, status, output hash and request identifier in a manifest. Treat partial/stale outputs as incomplete, lock each item during mutation, and invalidate downstream caches when dependencies change. Do not delete evidence blindly to retry an uncertain paid or external action: reconcile using the persisted request ID. Use a database/queue when concurrent writers or transactional requirements exceed file-based coordination.

Structured Output Design

When LLM outputs must be parsed programmatically, prompt design directly determines parsing reliability. The prompt must specify exact format requirements with examples.

Effective structure specification includes:

1. Section markers: Explicit headers or prefixes for parsing

2. Format examples: Show exactly what output should look like

3. Rationale disclosure: "I will be parsing this programmatically"

4. Constrained values: Enumerated options, score ranges, formats

Example prompt structure:

Analyze the following and provide your response in exactly this format:

## Summary
[Your summary here]

## Score
Rating: [1-10]

## Details
- Key point 1
- Key point 2

Follow this format exactly because I will be parsing it programmatically.

The parsing code must handle variations gracefully. LLMs do not follow instructions perfectly. Build parsers that:

• Use regex patterns flexible enough to handle minor formatting variations
• Reject or mark incomplete missing required fields; never invent a successful default score
• Log parsing failures for later review rather than crashing

Agent-Assisted Development

Modern agent-capable models can accelerate development significantly. The pattern is:

1. Describe the project goal and constraints

2. Let the agent generate initial implementation

3. Test and iterate on specific failures

4. Refine prompts and architecture based on results

This is about rapid iteration: generate, test, fix, repeat. The agent handles boilerplate and initial structure while you focus on domain-specific requirements and edge cases.

Key practices for effective agent-assisted development:

• Provide clear, specific requirements upfront
• Break large projects into discrete components
• Test each component before moving to the next
• Keep the agent focused on one task at a time

Cost and Scale Estimation

LLM processing has predictable costs that should be estimated before starting. The formula:

Total cost = input_tokens × input_rate + output_tokens × output_rate + reasoning/tool/media charges + retries + infrastructure

For batch processing:

• Estimate input tokens per item (prompt + context)
• Estimate output tokens per item (typical response length)
• Multiply by item count
• Add 20-30% buffer for retries and failures

Track actual costs during development. If costs exceed estimates significantly, re-evaluate the approach. Consider:

• Reducing context length through truncation
• Using smaller models for simpler items
• Caching and reusing partial results
• Parallel processing to reduce wall-clock time (not token cost)

Detailed Topics

Choosing Single vs Multi-Agent Architecture

Single-agent pipelines work for:

• Batch processing with independent items
• Tasks where items do not interact
• Simpler cost and complexity management

Multi-agent architectures work for:

• Parallel exploration of different aspects
• Tasks exceeding single context window capacity
• When specialized sub-agents improve quality

The primary reason for multi-agent is context isolation, not role anthropomorphization. Sub-agents get fresh context windows for focused subtasks. This prevents context degradation on long-running tasks.

See multi-agent-patterns skill for detailed architecture guidance.

Architectural Reduction

Start with minimal architecture. Add complexity only when proven necessary. Evaluate whether fewer tools improve the actual task; broader shell/SQL primitives can expand permissions and failure impact.

The linked Vercel case study is one environment-specific example, not a g

… (full content is in the download package)