Automate Org/Shop Scoping Audits For MaintenIX

Hey everyone, let's talk about something super important for our MaintenIX platform: making sure our organization and shop scoping is locked down tighter than a drum. You know, keeping everyone's data separate and secure is a huge deal, right? We've been working hard on our Feature-Role Matrix, and now it's time to make sure it's actually working as intended. That's where our new Audit Agent comes in, guys! This bad boy is going to automatically check every single change we push, making sure our org and shop scoping is on point across all MaintenIX features after we update that matrix on GitHub. It's all about catching issues early, preventing security headaches, and keeping our platform robust. We're talking about automating the heck out of this process to give us peace of mind and a super reliable system.

The Need for Speed and Security

So, why build this whole automated audit agent thing? Well, imagine this: we make a change to how roles and features interact, maybe we tweak a database query or update a frontend component. Without an automated check, it's super easy for a small mistake, like forgetting to filter by org_id in a query, to slip through. This could potentially expose sensitive data from one organization to another. Yikes! That's exactly the kind of nightmare we want to avoid. Our Feature-Role Matrix is designed to enforce these boundaries, but we need a way to verify that the implementation is correct. This audit agent acts as our vigilant security guard, constantly patrolling our codebase. It checks database queries, React Query keys, RLS policies, edge functions, frontend components, and even schema changes. If anything looks fishy, it flags it immediately. This means we catch problems before they hit production, saving us tons of debugging time and potential security breaches. Plus, think about the developer experience – getting instant feedback on a PR is way better than finding out about a critical issue days later. It’s about building quality and security right into our development workflow, making it easier for everyone to contribute securely. We're not just building features; we're building trust, and this audit agent is a massive step in that direction. It empowers us to move faster with confidence, knowing that our core security principles are being upheld automatically.

Step 1: The GitHub Actions Workflow - Our Automated Guardian

Alright, let's dive into the heart of our automation: the GitHub Actions Workflow. This .github/workflows/org-shop-audit.yml file is what kicks off our audit process every time code is pushed to our main or develop branches, or when a pull request is opened, synchronized, or reopened. Think of it as the central command for our security checks.

Here's the lowdown on what it does:

• Triggering the Audit: It listens for push events on main and develop, and for pull_request events like opened, synchronize, and reopened. This ensures that every significant code change is vetted.
• Environment Setup: It runs on ubuntu-latest, making sure we have a clean and consistent environment. It also sets up Node.js (version 18, specifically) and caches npm dependencies to speed things up.
• Code Checkout: The actions/checkout@v4 step fetches our repository code, including the full history (fetch-depth: 0) which might be needed for comparing changes.
• Dependency Installation: npm ci installs dependencies reliably, and npm install -g typescript @typescript-eslint/parser makes sure we have the necessary tools for our TypeScript audit script.
• Running the Audit Script: This is the main event! The scripts/audit-org-shop-scoping.js script is executed. We pass essential environment variables like GITHUB_TOKEN (for interacting with GitHub API), PR_NUMBER, COMMIT_SHA, and REPOSITORY to the script so it knows the context of the audit.
• Commenting on Pull Requests: If the audit is triggered by a pull_request, the actions/github-script@v7 step uses the GITHUB_TOKEN to post a detailed summary of the audit results directly as a comment on the PR. This gives developers immediate visibility into any issues found. The comment includes a summary (passed/failed, score, files scanned, issues found) and a collapsible section for detailed findings.
• Failing the Build: Crucially, if the audit script finds any critical issues (we'll get to how it defines these!), it will cause the job to fail (exit 1). This is our gatekeeper – no critical scoping violations can merge!

This workflow is the backbone of our automated security. It ensures that our org/shop scoping is checked consistently and automatically, providing real-time feedback and preventing insecure code from ever reaching our main branches. It’s all about making security a seamless part of our development process, guys!

Step 2: The Main Audit Script - The Brains of the Operation

Now, let's unpack the scripts/audit-org-shop-scoping.js script. This is where the actual magic happens! It's designed to meticulously scan our codebase for potential org/shop scoping violations. This script is the core logic that identifies where we might be missing filters, validation, or proper guards, ensuring data isolation is maintained.

Here’s a breakdown of what this script does:

• Initialization: It starts by setting up a results object to store all the audit findings, including pass/fail status, scores, counts of different issue types (critical, warnings, recommendations), files scanned, and detailed reports. It also defines patterns – a collection of regular expressions designed to sniff out common scoping mistakes across different parts of our application.
• Scoping Definitions: It maintains lists of orgScopedTables and shopScopedTables. These are crucial because the script uses these lists to know which database tables should have org_id or shop_id filters applied.
• Getting Changed Files: The getChangedFiles() method is smart! Instead of scanning the entire massive codebase every time, it uses git diff to figure out which files have actually been modified in the current commit or pull request. This makes the audit way faster. If it can't determine changed files (maybe it's run outside a Git context), it falls back to scanning all relevant files (getAllRelevantFiles()).
• Auditing Categories: The run() method orchestrates the auditing process by calling specific functions for different code categories:
  • auditDatabaseQueries(): This is a big one! It iterates through SQL files and TypeScript/JavaScript files that interact with the database. It specifically looks for queries on orgScopedTables that don't include an .eq('org_id', orgId) clause. It also checks if queries on shopScopedTables are missing a shop_id filter when an org_id filter is present, flagging these as potential issues.
  • auditReactQueries(): This checks our frontend code, particularly React components using React Query. It looks for queryKey definitions to ensure they include orgId when dealing with org-scoped data. It also verifies that enabled states for queries are properly dependent on orgId to prevent accidental data fetching.
  • auditRLSPolicies(): This dives into our Supabase schema files (.sql) to check if CREATE POLICY statements on org-scoped tables actually include org_id filtering. It also checks if org-scoped tables themselves are created with an org_id column.
  • auditEdgeFunctions(): It scans our serverless functions (like Supabase Edge Functions) to ensure they properly validate org_id parameters, user authentication (auth.getUser), and organization membership. Missing any of these is flagged as critical.
  • auditFrontendGuards(): On the frontend, this checks UI elements like buttons and links that perform sensitive actions (delete, create, update, etc.). It verifies that these actions are appropriately guarded by role checks (like <RequireRole> or canAccess) to prevent unauthorized users from performing critical operations.
  • auditSchemaChanges(): This specifically looks at migration files for ALTER TABLE statements. If an org_id or shop_id column is added to an org/shop-scoped table, it checks if corresponding RLS policies are also being updated or created.
• Issue Reporting: The addIssue() method is used throughout the script to record any violations found. It categorizes them as CRITICAL, WARNING, or RECOMMENDATION, updates the counts in the results object, and sets passed to false if any critical issues are found.
• Report Generation: Finally, generateReport() formats all the collected issues into a human-readable summary and a detailed breakdown, which is saved to audit-results.json. This file is then used by the GitHub Actions workflow to comment on the PR.

This script is the engine that drives our org/shop scoping compliance. By systematically checking these different areas, we can be much more confident that our data isolation is robust and secure.

Step 3: ESLint Plugin - IDE Integration for Real-time Linting

To make life even easier for our developers, we've created an ESLint plugin (eslint-plugin-org-shop-scoping.js). This isn't just for the automated checks; it provides real-time feedback directly in your IDE as you type! Imagine catching a scoping error the moment you write it, without even needing to commit or push. That's the power of IDE integration, guys.

This plugin introduces two new ESLint rules:

1. require-org-id-in-queries:

   • What it does: This rule targets Supabase queries (.from(...)) on tables that we've defined as being organization-scoped (like repair_orders, purchase_orders, etc.).
   • How it works: It analyzes the query chain to make sure there's an .eq('org_id', orgId) filter present. If it finds a query on an org-scoped table without this filter, ESLint will flag it as an error directly in your code editor.
   • Benefit: Prevents accidental data leaks by ensuring every query accessing org-specific data is properly scoped from the get-go. You'll see a red squiggle under the offending code immediately!

2. require-org-id-in-query-keys:

   • What it does: This rule focuses on React Query configurations, specifically the queryKey array.
   • How it works: It checks if the queryKey for data that should be organization-scoped actually includes an orgId or org_id identifier. It looks at the first element of the key (e.g., 'repair-orders') to infer if it's related to org-scoped data and then checks for the presence of orgId.
   • Benefit: Helps enforce the pattern of including orgId in query keys, which is crucial for caching and data fetching integrity. If you define a key like ['users'] for data that should be tied to an organization, this rule will warn you.

How to use it:

To enable these rules, you'll need to configure your ESLint setup (usually in an .eslintrc.js or .eslintrc.json file). You'd typically add the plugin and then specify the rules you want to enable, often as error or warn:

{
  "plugins": [
    "org-shop-scoping"
  ],
  "rules": {
    "org-shop-scoping/require-org-id-in-queries": "error",
    "org-shop-scoping/require-org-id-in-query-keys": "warn"
  }
}

(Note: The actual plugin file needs to be accessible, possibly by installing it as a local dependency or linking it.)

By integrating these checks directly into our IDEs via ESLint, we make it incredibly easy for developers to adhere to org/shop scoping best practices. It’s proactive security that happens while you code, making the entire process smoother and more secure for everyone involved.

Step 4: The Pre-commit Hook - Catching Issues Before They're Committed

Okay, so we've got our GitHub Actions workflow running audits on pull requests, and our ESLint plugin giving us instant feedback in the IDE. What else can we do to be super proactive about org/shop scoping? Enter the pre-commit hook! This little script, .husky/pre-commit, runs automatically right before Git stages your commit. It’s like a final checkpoint to ensure that no glaring scoping issues sneak into your commit history.

Here's how it works:

• Trigger: Whenever you run git commit, Husky (our hook management tool) automatically executes the script defined in .husky/pre-commit.
• Running the Audit: The script executes node scripts/audit-org-shop-scoping.js --mode=pre-commit. This command runs our main audit script, but potentially in a mode optimized for pre-commit checks (though in this example, it’s mostly calling the script). The key here is that it runs the same audit logic we use in CI.
• Staging Check: The script checks the exit code ($?) of the audit script. If the audit finds any issues (indicated by a non-zero exit code), it means something is wrong.
• Failure and Guidance: If the audit fails, the pre-commit hook prints an error message: ❌ Org/shop scoping audit failed! and provides a helpful tip: 💡 Run 'npm run audit:org-shop' to see detailed results. It then exits with a status code of 1, which prevents the commit from being created.
• Success: If the audit passes (exit code 0), the hook allows the commit to proceed. You'll see a success message: ✅ Org/shop scoping audit passed!

Why is this so important, guys?

• Immediate Feedback: You find out about a scoping problem immediately after running git commit, not hours later when a CI pipeline fails or, worse, after it's merged.
• Reduced CI Load: By catching issues early, we reduce the number of failed CI runs, saving valuable build minutes.
• Developer Discipline: It encourages developers to fix issues right away while the code is fresh in their minds, rather than deferring them.
• Consistency: Ensures that the same checks are run locally as are run in our CI environment.

Setting up this pre-commit hook means we're adding another layer of defense against org/shop scoping violations. It empowers developers to take ownership of code quality and security right at the source – their local machine, before sharing it with the team. It’s a small step that makes a huge difference in maintaining a secure and well-structured codebase.

Step 5: Package.json Scripts - Streamlining the Process

To make all these commands easy to run and manage, we've added some handy scripts to our package.json file. These scripts act as shortcuts, allowing us to execute the audit and linting commands with simple, memorable names. It's all about making the workflow as smooth as possible for everyone, guys!

Here are the key scripts we've added:

• audit:org-shop:

  • Command: node scripts/audit-org-shop-scoping.js
  • Purpose: This is your go-to command for running the full org/shop scoping audit manually. If you want to see the detailed report outside of a PR comment or pre-commit check, just run npm run audit:org-shop in your terminal.

• audit:org-shop:fix:

  • Command: node scripts/audit-org-shop-scoping.js --fix
  • Purpose: This script attempts to automatically fix some of the identified scoping issues. While not all problems can be auto-fixed, this can save a lot of time by correcting common mistakes. Run npm run audit:org-shop:fix to apply any auto-fixable changes.

• lint:org-shop:

  • Command: eslint --ext .ts,.tsx . --rule 'org-shop-scoping/require-org-id-in-queries: error'
  • Purpose: This script specifically runs the ESLint check for the require-org-id-in-queries rule across all TypeScript and TypeScriptX files in your project. It's configured here to report violations as errors, making them highly visible. This is great for quickly checking your code against this specific rule outside of the full audit script or IDE integration.

How these scripts help:

• Simplicity: Instead of remembering complex Node.js commands, you just type npm run <script-name>.
• Consistency: Ensures everyone on the team is running the audit and linting commands in the exact same way.
• Accessibility: Makes it easy to trigger the audit process, whether for manual checks, attempting fixes, or focused linting.
• Integration: These scripts are what the pre-commit hook and GitHub Actions workflow rely on to execute the core audit logic.

By standardizing these commands in package.json, we make sure that running org/shop scoping checks is an intuitive and accessible part of our daily development routine. It’s all about reducing friction and making security a seamless part of the coding experience.

Expected Outcomes: A More Secure and Robust MaintenIX

So, what do we gain from implementing this whole automated org/shop scoping audit system? The outcomes are pretty significant and touch on several key areas of our development process and platform integrity. We're not just adding a tool; we're fundamentally improving how we build and maintain MaintenIX. Let's break down the expected wins, guys!

• Automated Security Checks on Every PR: This is the big one. Every single time a pull request is opened or updated, our audit agent springs into action. It scans the changes for any violations of our org/shop scoping rules. This means potential security vulnerabilities related to data isolation are identified before they ever get a chance to be merged into our main codebase. It’s like having a tireless security expert reviewing every line of code.

• Real-time Feedback for Developers: Gone are the days of waiting for a long CI build to fail just to find out about a simple scoping mistake. Our GitHub Actions comments and IDE integrations (via ESLint) provide immediate feedback. Developers know right away if they’ve missed an org_id filter or a role guard. This speeds up the development cycle significantly because issues are addressed while the code is fresh in the developer's mind.

• Consistent Compliance Tracking: The audit produces detailed reports, including a compliance score and breakdowns of issues found. This data can be tracked over time. We can see trends in our org/shop scoping compliance, identify areas where we might be struggling, and measure the effectiveness of our efforts. This provides valuable insights for process improvement and team training.

• Prevention Through Pre-commit Hooks: The pre-commit hook acts as our first line of defense. By running the audit before a commit is even finalized, we prevent many common scoping mistakes from entering our version control history in the first place. This fosters better coding habits and reduces the likelihood of introducing bugs.

• Developer Education and Best Practices: The detailed suggestions and examples provided by the audit script serve as a learning tool. When a violation is flagged, the audit doesn't just say