Process Documentation: Industry Standards & Best Practices

This guide covers industry-standard approaches for documenting backend processes and how to automate documentation generation from code comments.

---

🎯 Overview

Process documentation helps non-developer teams understand what happens when they trigger actions in the system. This is especially important for:

• Transparency - Non-dev teams need to see what happens behind the scenes
• Troubleshooting - Understanding the full flow helps diagnose issues
• Onboarding - New team members can understand complex workflows
• Compliance - Documented processes support audit requirements

---

📊 Industry Standard Approaches

1. Code-First Documentation (✅ What We Use)

Description: Structured comments in code that are automatically extracted and converted to documentation.

Pros:

• ✅ Documentation lives with code (single source of truth)
• ✅ Version-controlled alongside code changes
• ✅ Developer-friendly workflow
• ✅ Can be validated in CI/CD
• ✅ Stays close to implementation

Cons:

• ⚠️ Requires developer discipline
• ⚠️ Can get out of sync if not maintained
• ⚠️ Requires custom tooling

Tools:

• JSDoc/TSDoc with custom tags
• YAML manifests for process definitions
• Custom documentation generators

Best For: Teams that want documentation tightly coupled with code.

---

2. OpenAPI/Swagger (✅ What We Use for APIs)

Description: Standard specification for REST API documentation, auto-generated from code annotations.

Pros:

• ✅ Industry standard
• ✅ Interactive documentation (try endpoints)
• ✅ Auto-generated from code
• ✅ Tooling ecosystem (Redoc, Swagger UI)
• ✅ Client code generation

Cons:

• ⚠️ Only covers API endpoints
• ⚠️ Doesn't document internal processes
• ⚠️ Limited to request/response flows

Best For: API documentation and external integrations.

---

3. Architecture Decision Records (ADRs)

Description: Markdown files documenting why architectural decisions were made.

Structure:

docs/adr/
  0001-use-strapi-for-cms.md
  0002-postgresql-database-choice.md
  0003-process-documentation-system.md

Pros:

• ✅ Captures decision context
• ✅ Historical record
• ✅ Helps future developers understand "why"

Cons:

• ⚠️ Not process-focused
• ⚠️ More about decisions than workflows

Best For: Documenting architectural choices and their rationale.

---

4. Sequence Diagrams from Code

Description: Visual flow diagrams generated from code annotations (Mermaid, PlantUML).

Example:

/**
 * @sequence-diagram
 * User->Frontend: Click button
 * Frontend->Backend: API call
 * Backend->Database: Save data
 * Backend->Email: Send notification
 */

Pros:

• ✅ Visual representation
• ✅ Easy to understand flow
• ✅ Can be auto-generated

Cons:

• ⚠️ Can be verbose
• ⚠️ Requires diagram syntax knowledge
• ⚠️ May not capture all details

Best For: Visual process flows and team presentations.

---

5. Documentation-as-Code (Markdown in Repo)

Description: Markdown files in repository that are manually maintained.

Pros:

• ✅ Version-controlled
• ✅ Easy to edit
• ✅ Collaborative (PRs)

Cons:

• ⚠️ Can drift from code
• ⚠️ Manual maintenance burden
• ⚠️ Often outdated

Best For: High-level documentation, guides, tutorials.

---

6. External Documentation Platforms

Description: Tools like Confluence, Notion, GitBook, or dedicated wikis.

Pros:

• ✅ Rich formatting
• ✅ Easy for non-devs
• ✅ Search capabilities
• ✅ Comments and collaboration

Cons:

• ⚠️ Separate from code
• ⚠️ Often becomes outdated
• ⚠️ No automatic sync
• ⚠️ Additional tooling cost

Best For: High-level business documentation, not technical processes.

---

🏆 Recommended Approach: Hybrid System

Best Practice: Combine multiple approaches for comprehensive coverage:

1. Code Comments → Process documentation (what we have)
2. OpenAPI → API reference (what we have)
3. ADRs → Architecture decisions (optional)
4. Markdown Guides → High-level overviews (what we have)

---

🔄 Our Current System

We use a three-layer approach:

Layer 1: Code Comments

Structured JSDoc-style comments in source code:

/**
 * @process-id PROGRAM-CONFIGURATION-HANDOVER
 * @process-step 2.1
 * @repo strapi-docker
 * @description Automatically tracks configuration changes before save
 * @implicit Automatic Strapi lifecycle hook
 * @triggers PROGRAM-CONFIGURATION-HANDOVER:2.2
 */

Layer 2: Process Manifests

YAML files defining complete workflows:

process:
  id: PROGRAM-CONFIGURATION-HANDOVER
  name: Program Configuration Handover
  description: |
    Complete workflow for transferring program configuration
    from Sales team to Learning team
  owner:
    - team-sales
    - team-learning
  repositories:
    - platform
    - strapi-docker

steps:
  - id: "1.0"
    type: user-action
    repo: platform
    description: User clicks "Handover Configuration" button
    explicit: true

Layer 3: Generated Documentation

Auto-generated Docusaurus pages with:

• Step-by-step breakdowns
• Mermaid sequence diagrams
• Code references
• Side effects documentation
• Error handling

---

🚀 Automating Documentation Generation

Current State

✅ Manual Generation:

cd documentation
node scripts/generate-process-docs.js

Recommended: Automatic on Merge

Add CI/CD automation to generate and commit documentation automatically when code changes.

---

📝 Implementation: Auto-Generate on Merge

Option 1: Backend Repository Workflow (Recommended)

Create a workflow in strapi-docker/.github/workflows/update-process-docs.yml:

name: Update Process Documentation

on:
  push:
    branches:
      - main
      - develop
    paths:
      - 'src/**/*.js'
      - 'src/**/*.ts'
      - 'documentation/process-registry/**/*.yml'

jobs:
  update-docs:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Backend Repository
        uses: actions/checkout@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          fetch-depth: 0

      - name: Checkout Documentation Repository
        uses: actions/checkout@v4
        with:
          repository: DeepSkill/documentation
          token: ${{ secrets.DOCUMENTATION_REPO_TOKEN }}
          path: documentation

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
          cache-dependency-path: documentation/package-lock.json

      - name: Install Documentation Dependencies
        run: |
          cd documentation
          npm ci

      - name: Detect Changed Process Files
        id: detect
        run: |
          cd documentation
          
          # Find changed YAML files in process-registry
          CHANGED_FILES=$(git diff --name-only HEAD~1 HEAD | grep 'process-registry/.*\.yml$' || true)
          
          if [ -z "$CHANGED_FILES" ]; then
            # Check if any code files changed that might have process comments
            CODE_CHANGES=$(git diff --name-only HEAD~1 HEAD | grep -E '\.(js|ts|jsx|tsx)$' || true)
            if [ -n "$CODE_CHANGES" ]; then
              echo "Code files changed - regenerating all process docs"
              echo "regenerate_all=true" >> $GITHUB_OUTPUT
            else
              echo "No process-related changes detected"
              echo "regenerate_all=false" >> $GITHUB_OUTPUT
            fi
          else
            echo "Process manifest files changed"
            echo "changed_files<<EOF" >> $GITHUB_OUTPUT
            echo "$CHANGED_FILES" >> $GITHUB_OUTPUT
            echo "EOF" >> $GITHUB_OUTPUT
            echo "regenerate_all=false" >> $GITHUB_OUTPUT
          fi

      - name: Generate Process Documentation
        id: generate
        run: |
          cd documentation
          
          if [ "${{ steps.detect.outputs.regenerate_all }}" = "true" ]; then
            echo "🔄 Regenerating all process documentation..."
            node scripts/generate-process-docs.js
            echo "regenerated=true" >> $GITHUB_OUTPUT
          else
            # Extract process IDs from changed files
            CHANGED_FILES="${{ steps.detect.outputs.changed_files }}"
            if [ -n "$CHANGED_FILES" ]; then
              for file in $CHANGED_FILES; do
                PROCESS_ID=$(basename "$file" .yml | tr '[:lower:]' '[:upper:]')
                echo "🔄 Regenerating documentation for: $PROCESS_ID"
                node scripts/generate-process-docs.js "$PROCESS_ID"
              done
              echo "regenerated=true" >> $GITHUB_OUTPUT
            else
              echo "No documentation to regenerate"
              echo "regenerated=false" >> $GITHUB_OUTPUT
            fi
          fi

      - name: Commit and Push Documentation
        if: steps.generate.outputs.regenerated == 'true'
        run: |
          cd documentation
          
          git config --local user.email "action@github.com"
          git config --local user.name "GitHub Action"
          
          git add docs/internal/processes/detailed/
          
          if git diff --staged --quiet; then
            echo "ℹ️ No documentation changes to commit"
          else
            git commit -m "docs: auto-update process documentation

            Source: ${{ github.repository }}@${{ github.sha }}
            Branch: ${{ github.ref_name }}
            
            Auto-generated from process manifests and code comments."
            git push
            echo "✅ Documentation updated"
          fi

      - name: Create Job Summary
        if: always()
        run: |
          echo "## 📚 Process Documentation Update" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "**Source:** ${{ github.repository }}@${{ github.sha }}" >> $GITHUB_STEP_SUMMARY
          echo "**Branch:** ${{ github.ref_name }}" >> $GITHUB_STEP_SUMMARY
          
          if [ "${{ steps.generate.outputs.regenerated }}" = "true" ]; then
            echo "✅ Process documentation regenerated" >> $GITHUB_STEP_SUMMARY
          else
            echo "ℹ️ No documentation changes needed" >> $GITHUB_STEP_SUMMARY
          fi

Option 2: Documentation Repository Workflow

Alternatively, trigger from documentation repository when process manifests change:

name: Generate Process Documentation

on:
  push:
    branches:
      - main
    paths:
      - 'process-registry/**/*.yml'
      - 'scripts/generate-process-docs.js'

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - run: npm ci
      
      - name: Generate All Process Docs
        run: node scripts/generate-process-docs.js
      
      - name: Commit Generated Docs
        run: |
          git config user.name "GitHub Action"
          git config user.email "action@github.com"
          git add docs/internal/processes/detailed/
          git diff --staged --quiet || git commit -m "docs: auto-update process documentation"
          git push

---

🔍 Process Comment Extraction (Advanced)

For even more automation, extract process comments directly from code:

Script: Extract Process Comments

// scripts/extract-process-comments.js
const fs = require('fs');
const path = require('path');
const { execSync } = require('child_process');

/**
 * Extracts @process-* comments from code files
 */
function extractProcessComments(repoPath) {
  const processComments = [];
  
  // Find all JS/TS files
  const files = execSync(
    `find ${repoPath} -type f \\( -name "*.js" -o -name "*.ts" -o -name "*.jsx" -o -name "*.tsx" \\)`,
    { encoding: 'utf-8' }
  ).trim().split('\n');
  
  files.forEach(file => {
    const content = fs.readFileSync(file, 'utf-8');
    const lines = content.split('\n');
    
    let inProcessComment = false;
    let commentBlock = [];
    let lineNumber = 0;
    
    lines.forEach((line, idx) => {
      if (line.includes('@process-id')) {
        inProcessComment = true;
        lineNumber = idx + 1;
        commentBlock = [line];
      } else if (inProcessComment) {
        commentBlock.push(line);
        if (line.includes('*/')) {
          // Parse comment block
          const processInfo = parseProcessComment(commentBlock.join('\n'));
          if (processInfo) {
            processComments.push({
              ...processInfo,
              file: path.relative(repoPath, file),
              line: lineNumber
            });
          }
          inProcessComment = false;
          commentBlock = [];
        }
      }
    });
  });
  
  return processComments;
}

function parseProcessComment(comment) {
  const processId = comment.match(/@process-id\s+(\S+)/)?.[1];
  const step = comment.match(/@process-step\s+(\S+)/)?.[1];
  const repo = comment.match(/@repo\s+(\S+)/)?.[1];
  const description = comment.match(/@description\s+(.+?)(?:\n|@|$)/s)?.[1]?.trim();
  
  if (!processId || !step) return null;
  
  return {
    processId,
    step,
    repo,
    description
  };
}

// Usage
if (require.main === module) {
  const repoPath = process.argv[2] || process.cwd();
  const comments = extractProcessComments(repoPath);
  console.log(JSON.stringify(comments, null, 2));
}

module.exports = { extractProcessComments };

---

✅ Best Practices

1. Keep Comments Close to Code

• Document processes where they happen
• Use consistent process IDs across repos
• Link steps with @triggers and @triggered-by

2. Automate Everything

• Generate docs on merge (CI/CD)
• Validate process IDs in PR checks
• Alert when docs are outdated

3. Make It Visible

• Include in PR descriptions
• Link from error messages
• Add to team dashboards

4. Keep It Simple

• Don't document every function
• Focus on multi-step processes
• Document implicit actions (they're the most important)

5. Regular Maintenance

• Quarterly review of all process docs
• Remove outdated processes
• Update when code changes

---

🛠️ Validation & Quality Checks

Pre-commit Hook

Add validation to ensure process comments are valid:

#!/bin/bash
# .husky/pre-commit

# Check for process comments without manifests
node scripts/validate-process-docs.js

PR Checks

Validate in CI/CD:

- name: Validate Process Documentation
  run: |
    node scripts/validate-process-docs.js
    node scripts/check-process-ids.js

---

📚 Related Documentation

• Process Documentation Standards
• Process Registry
• API Reference

---

🎓 Industry Examples

Companies Using Similar Approaches

1. Stripe - Extensive API documentation with code examples
2. GitHub - OpenAPI specs for all APIs
3. Shopify - Comprehensive process documentation
4. Atlassian - ADRs for architecture decisions

Open Source Projects

• Kubernetes - Extensive process documentation
• React - RFC process with detailed documentation
• Rust - Comprehensive internal process docs

---

🔗 Additional Resources

• JSDoc Documentation
• OpenAPI Specification
• Mermaid Diagrams
• Documentation as Code

---

❓ Questions?

Contact the platform team or documentation maintainers for guidance on process documentation.