quodeq, evaluated by quodeq

Fault Tolerance · R-FT-14

The JSON-RPC server reads requests from its stdio client, but this handler discards each malformed JSON line and sends no parse-error response. The caller receives no response for that request and blocks or times out instead of learning that its request was invalid.

25        line = line.strip()26        if not line:27            continue28        try:29            obj = json.loads(line)30        except json.JSONDecodeError:31            continue32        if isinstance(obj, dict):33            return obj34    return None35 

Fault Tolerance · R-FT-4

`json.loads` accepts valid JSON arrays, but this code immediately calls `obj.get` without checking that the result is a dict. A provider stream containing a text line such as `[]` reaches this parser and raises AttributeError, aborting the live findings response with a server error instead of skipping the malformed finding row.

28        return None29    try:30        obj = json.loads(stripped_line)31    except json.JSONDecodeError:32        return None33    if not obj.get("p") or obj.get("t") not in FINDING_TYPES:34        return None35    key = f"{obj['p']}:{obj.get('file', '')}:{obj.get('line', '')}:{obj['t']}"36    if key in seen:37        return None38    seen.add(key)

Fault Tolerance · R-FT-4

`json.loads` accepts valid JSON arrays and scalars, but this line calls `.get` without requiring a mapping. If `repository_info.json` contains a valid non-object value, `_resolve_repo_source` raises `AttributeError` before the endpoint's fallback handler and the estimates request returns an error instead of the documented zero payload.

34    info_path = project_dir / "repository_info.json"35    try:36        info = json.loads(info_path.read_text(encoding="utf-8"))37    except (OSError, json.JSONDecodeError):38        return None39    src = Path(info.get("path") or "")40    if not info.get("path") or not src.is_dir():41        return None42    return src, info.get("scopePath") or None43 44 

Fault Tolerance · R-FT-4

The expression `etype = data.get("type", "unknown")` assumes `json.loads(line)` produced a mapping, but valid JSON lines such as `[]`, `null`, or `"text"` produce other types and are not rejected by the preceding `JSONDecodeError` handler. An unexpected scalar from the CLI stream raises `AttributeError` here and aborts parsing the entire evidence stream, failing that evaluation instead of skipping the invalid event.

76            try:77                data = json.loads(line)78            except json.JSONDecodeError:79                continue80 81            etype = data.get("type", "unknown")82            extractor = TEXT_EXTRACTORS.get(etype)83            if extractor:84                _process_texts(extractor(data), out, stats)85                files_read.update(_collect_file_reads(data))86 

Reusability · M-REU-2

cached_skills returns the dictionary stored in the module-level memo directly. Any caller that edits the returned mapping changes the result seen by every later caller for the same file stamp, leaving the cache inconsistent with the skill files.

123    directory = skills_dir or skills_directory(env)124    stamp = _skills_stamp(directory)125    with _SKILLS_MEMO_LOCK:126        hit = _SKILLS_MEMO.get(directory)127        if hit is not None and hit[0] == stamp:128            return hit[1]129    skills = load_skills(directory)130    with _SKILLS_MEMO_LOCK:131        _SKILLS_MEMO[directory] = (stamp, skills)132    return skills133 

Reusability · M-REU-4

remember_run_dates also resolves the index database from process environment with no env argument, preventing a caller from injecting its configuration and isolating the operation.

74def remember_run_dates(project: str, dates: dict[str, tuple[str | None, str]]) -> None:75    """Store the dates ``parse_run_date`` resolved for *project*'s runs; best-effort."""76    if not dates:77        return78    try:79        db = open_index(Path(get_index_db_path()))80        try:81            write_run_dates(db, project, dates)82        finally:83            db.close()84    except (sqlite3.Error, OSError):

Reusability · M-REU-4

resolve_shell accepts env and reads COMSPEC from that mapping, but shutil.which(cand) searches the process PATH instead of src['PATH']. Callers supplying an isolated environment therefore still launch a shell selected from the host process, making shell resolution non-deterministic and preventing tests or embedders from controlling it.

18# keeps the read promptly stoppable (the reader loop re-checks its stop flag19# each cycle instead of parking).20_EMPTY_READ_SLEEP_S = 0.0521 22 23def resolve_shell(env: dict[str, str] | None = None) -> str:24    src = resolve_env(env)25    for cand in ("pwsh", "powershell"):26        found = shutil.which(cand)27        if found:28            return found29    return src.get("COMSPEC", "cmd.exe")30 31 32class WindowsPty:33    def __init__(self, argv: list[str] | None = None, env: dict[str, str] | None = None):

Modularity · M-MOD-4

The component accepts nine separately named inputs, exceeding the six parameter limit; each caller must keep a wide positional prop contract synchronized, and changes to state inputs require updating that large signature and forwarding list.

68// A failed fetch with nothing to show must render as an error, not the69// "no evaluations yet" empty state -- otherwise a 404/500/timeout tells70// the user their existing evaluations are gone. While a retry is in71// flight (error still set, isFetching true), show the loader instead so72// clicking Retry visibly does something.73export function MapNoDimensionsState({ loading, error, isFetching, selectedSource, selectedProject, projectName, isRefreshing, onNavigate, onRetry }) {74  if (loading) return <MapLoadingState />;75  if (error) return isFetching ? <MapLoadingState /> : <MapErrorState error={error} onRetry={onRetry} />;76  return (77    <MapNoEvaluationsState78      selectedSource={selectedSource} selectedProject={selectedProject} projectName={projectName}

Capacity · P-CAP-2

`result = run_llamacpp_concurrency_test(model)` runs the slow concurrency probe synchronously in the route. The request worker remains occupied until inference finishes, reducing API request capacity.

95    whatever it was started with.96    """97    _, model, err = model_request(require_nonempty=False)98    if err is not None:99        return err100    result = run_llamacpp_concurrency_test(model)101    return jsonify(result)102 103 104def omlx_status() -> Response:105    """Report whether the omlx server at ``?base_url=`` is reachable.

Capacity · P-CAP-4

Each accepted `/messages` request starts a fresh daemon `threading.Thread`, while `AssistantTurnState` limits only SSE streams and does not cap concurrent turns. Concurrent requests from distinct sessions therefore allocate threads without a process-wide bound; sustained slow model calls exhaust the server's thread and memory capacity.

63        try:64            run_turn(turn, repository=repo, tool_ctx=tool_ctx, cancel=cancel)65        finally:66            state.release_turn(turn.session_id)67 68    threading.Thread(target=_worker, daemon=True).start()69 70 71def _sse_release_guard(state: AssistantTurnState):72    """Idempotent release-once wrapper around ``state.close_sse_stream()``.73 

Resource Utilisation · P-RES-3

read_text() loads each complete actions log, and splitlines() plus the accumulated lines list retain additional copies while merging. As the append-only logs grow, a merge requires memory proportional to all source log contents.

61    seen: set[str] = set()62    lines: list[str] = []63    for source in srcs:64        if not source.exists():65            continue66        for raw in source.read_text(encoding="utf-8").splitlines():67            line = raw.strip()68            if line and line not in seen:69                seen.add(line)70                lines.append(line)71    if not lines:

Capacity · P-CAP-2

`scan_project(...)` completes before the HTTP response, keeping the Flask request worker occupied for the full filesystem scan; scanning a large local project ties up a worker and delays other requests.

125    if cached is not None:126        return cached127    project_path, err = _local_scan_root(project_dir)128    if err is not None:129        return err130    return _scan_response(scan_project(project_path, output_dir=project_dir))131 132 133def project_estimates(project: str) -> Response | tuple[Response, int]:134    """Return read-only pre-run per-dimension file estimates for a project.135 

User Error Protection · U-UEP-4

ReqRef requires both constructor arguments syntactically, but its documented model-building path allows empty strings for both fields. A reference with no label or URL is therefore represented as valid typed data and downstream users cannot rely on these required fields being usable.

10 11    Built by ``core.finding_mappings._coerce_req_refs`` from whatever the12    model emitted, so both fields can be empty strings.13    """14 15    label: str16    url: str17 

User Error Protection · U-UEP-1

The root boundary check accepts any sibling path sharing the same string prefix. With rootPath `/srv/project`, navigating to `/srv/project-old` passes and the folder browser displays files outside the selected root, so this validation fails to protect the user's chosen scope.

36  }37  const navigation = { setLoading, setNavError, updateNavState };38 39  const navigate = useCallback((path) => {40    // Prevent navigating above rootPath when rootPath is set41    if (rootPath && path && !path.startsWith(rootPath)) {42      setPathInput(rootPath);43      return;44    }45    // Fire-and-forget: navigateFolder already catches its own errors and46    // sets navError; callers (click handlers, the mount effect below) are

Appropriateness Recognisability · U-APR-1

When a shared-project reference has no run ID, this query always calls api.getFindingDetail, ignoring ref.source. The shared project ID is therefore sent to the local endpoint, so deferred compliance detail on accumulated shared views fails to load and is marked unavailable.

16      queryFn: () => fetchDetail(ref.project, { kind, dimension: ref.dimension, run: ref.run, ...scope }),17    };18  }19  return {20    queryKey: projectKeys.findingDetail(ref.project, ref.asOf, kind, ref.dimension, ref.generation, scope),21    queryFn: () => api.getFindingDetail(ref.project, { kind, dimension: ref.dimension, asOf: ref.asOf, ...scope }),22  };23}24 25/**26 * Items of one kind with the detail /scores or /scores/<run> deferred

Accessibility · U-ACC-1

The explicit `aria-label` overrides the button's descendant text and defaults to only `label`; screen-reader users activating or reviewing a clickable stat hear its category (for example, “SCORE”) without the displayed score or count.

64    // breaks hydration, and is an accessibility trap. The button's ::after is65    // stretched over the whole card in CSS so the full square is the click66    // target; severity badges remain independent siblings raised above it.67    return (68      <div className={className}>69        <button type="button" className="term-stat__hitbox" onClick={onClick} aria-label={ariaLabel || label}>70          {labelValue}71        </button>72        {hintEl}73      </div>74    );

Adaptability · F-ADP-1

DASHBOARD_BASE_PORT is hardcoded as 7863 while the dashboard also accepts QUODEQ_DASHBOARD_PORT at runtime. A deployment that changes the configured port leaves this generated port allowlist tied to the default, so the API CSP does not adapt to the active dashboard port.

15out of scope here and stays uncovered by connect-src beyond 'self' (not16widened to dashboard/_networking.py's MAX_PORT_SCAN_TRIES).17"""18from __future__ import annotations19 20DASHBOARD_BASE_PORT = 786321PORT_SCAN_SPAN = 522 23 24def alt_ports() -> tuple[int, ...]:25    """Return the dashboard alt ports the API's CSP allows explicitly."""

Replaceability · F-RPL-1

This parser imports the concrete SqliteFindingsRepository and constructs it directly in load_evidence_map_from_db, so replacing the findings backend requires editing this parser instead of supplying an implementation through a persistence boundary.

12from quodeq.core.types.finding_type import FindingType13from quodeq.data.fs.report_parser._date_utils import find_date_in_dir, normalize_date14from quodeq.data.fs.report_parser._run_info import safe_read_dir15from quodeq.data.fs.report_parser.finding_response import finding_to_response_dict16from quodeq.data.sqlite.connection import EVALUATION_DB_FILENAME17from quodeq.data.sqlite.findings_repository import SqliteFindingsRepository18from quodeq.shared.constants import EVIDENCE_DIRNAME, MANIFEST_FILENAME19from quodeq.shared.utils import read_json20 21_logger = logging.getLogger(__name__)22 

Replaceability · F-RPL-1

Although `DeduplicationStore` is declared as an abstraction, the constructor assigns a concrete set and exposes no collaborator parameter. Replacing deduplication storage requires modifying FindingsRouter instead of substituting an implementation.

107        file_reader: FileReader | None = None,108        event_log: "EventLogWriter | None" = None,109        on_file_done: "Callable[[str, list[dict]], None] | None" = None,110    ):111        self._fh = output_fh112        self._enricher = FindingEnricher(context or CompiledContext(), file_reader, log=SHARED_LOG)113        self._seen: DeduplicationStore = set()114        self._event_log: EventLogWriter | None = event_log115        self._on_file_done: "Callable[[str, list[dict]], None] | None" = on_file_done116        self._findings_by_file: dict[str, list[dict]] = {}117        # (file, line, reported req) refused once: the model gets one retry,

Separation of Concerns · CLEA-SEP-03

`evaluators_dir.is_dir()` and `evaluators_dir.glob("*.json")` make the service layer depend on directory layout and filesystem access. A change to the standards storage adapter requires changing this service as well.

53"""Query operations for listing and retrieving standards."""54from __future__ import annotations55 56import logging57from collections.abc import Callable58from pathlib import Path59 60from quodeq.core.types.standard import StandardDetail, StandardMeta61from quodeq.services._standards_io import (62    build_builtin_detail, build_builtin_meta, build_custom_meta, build_detail,63    count_principles_and_requirements, get_builtin_weight, is_builtin_id,64    load_cwe_entries,65)66from quodeq.shared.validation import validate_path_segment67 68logger = logging.getLogger(__name__)69 70 71def list_builtin(dimensions_file: Path, compiled_dir: Path,72                 read_json: Callable) -> list[StandardMeta]:73    """Return metadata for all built-in standards."""74    try:75        data = read_json(dimensions_file)76    except (OSError, ValueError) as exc:77        logger.warning("Cannot read dimensions file: %s", exc)78        return []79    out: list[StandardMeta] = []80    for dim in data.get("applies", []):81        try:82            p_count, r_count, description, version = _read_compiled_meta(dim["id"], compiled_dir, read_json)83            out.append(build_builtin_meta(dim, p_count, r_count, description, version))84        except (OSError, ValueError, KeyError) as exc:85            logger.warning("Skipping invalid dimension entry %r: %s", dim, exc)86    return out87 88 89def _read_compiled_meta(dimension_id: str, compiled_dir: Path,90                        read_json: Callable) -> tuple[int, int, str, str | None]:91    """Return (principle_count, requirement_count, description, version) for *dimension_id*."""92    path = compiled_dir / f"{dimension_id}.json"

Use Case Encapsulation · CLEA-USE-05

The use case imports concrete `data.fs` loaders and `services.wiring` filesystem operations and calls them throughout scoring. This couples cancellation scoring to the filesystem implementation, so changing persistence requires editing the use case.

24"""Use case: score completed evidence after cancellation.25 26Extracted from ``evaluation_mixin`` so the API layer has a public entry27point instead of importing a private helper.28"""29from __future__ import annotations30 31import json32import logging33import time34from collections.abc import Callable35from dataclasses import dataclass36from pathlib import Path37from typing import Any38 39from quodeq.config.evidence_env import cwe_url_template40from quodeq.config.services_env import cancel_escalation_window_s41from quodeq.core.evidence.parser import (42    EvidenceContext, EvidenceParseOptions, parse_jsonl_to_evidence)43from quodeq.core.run.dimensions import DimState44from quodeq.core.run.job_status import JobStatus45from quodeq.core.run.state import STATUS_FILENAME46from quodeq.core.scoring.params import ScoringParams47from quodeq.data.fs.standards_loader import load_compiled_refs, read_req_to_principle_map48from quodeq.core.scoring.engine import score_evidence49from quodeq.core.utils.io import resolve_child_dir50from quodeq.services.scored_jobs_registry import ScoringClaims51from quodeq.services.background import BackgroundRunner52from quodeq.services.grade_formula import load_params53from quodeq.shared.fault_isolation import run_isolated54from quodeq.shared.log_sink import log_malformed_jsonl_line, log_quarantined_findings55from quodeq.services.wiring import (56    dimension_queue_file,57    dimension_report_exists,58    ensure_dir,59    list_dimension_evidence,60    queue_file_exists,61    read_dimensions,62    read_queue_files_count,63    read_scan_total_files,

Use Case Encapsulation · CLEA-USE-05

The dimension workflow imports and constructs `FileQueue` directly while preparing dispatch. Queue persistence is a concrete filesystem adapter dependency inside orchestration, so changing queue storage requires changing the subagent workflow.

32"""Subagent processing path -- runs a dimension via N parallel subagents.33 34The verify-pool is gone. V2's content-addressed cache35already invalidates on file/standards/prompts changes, triggering a36full fresh dispatch. The V1 "carry-forward + verify-when-rules-change"37optimization no longer earns its complexity.38"""39from __future__ import annotations40 41from dataclasses import dataclass42from pathlib import Path43from typing import Any, Callable44 45from quodeq.analysis.run_types import RunConfig, AnalysisContext46from quodeq.core.evidence.model import Evidence47from quodeq.core.observability import NULL_LOG, LogSink48 49# Re-exports from split modules -- keep the public API stable50from quodeq.analysis.subagents.source_files import list_source_files51from quodeq.analysis.subagents._prompts import build_subagent_prompt52from quodeq.analysis.subagents._pool_launcher import (  # noqa: F40153    LaunchPoolParams,54    compute_files_per_agent,55    default_subagent_model,56    launch_pool,57    collect_all_evidence,58)59from quodeq.analysis.subagents._evidence_collector import (60    CollectionContext,61    collect_evidence,62)63from quodeq.analysis.subagents.file_queue import FileQueue64from quodeq.analysis.subagents._consolidated import (65    process_consolidated_dimensions as _process_consolidated_impl,66)67 68 69@dataclass70class DimensionCallbacks:71    """Collaborators a dimension dispatch is handed: the single-agent

Dependency Rule · CLEA-DEP-01

The analysis namespace imports LocalFileBackend from quodeq.data.cache_store, making the inner analysis package depend on an outer storage implementation. Replacing or relocating the cache adapter therefore requires preserving a reverse dependency in the analysis layer.

5This module re-exports the public names so every pre-existing6``quodeq.analysis.cache.local`` import path keeps working.7"""8from __future__ import annotations9 10from quodeq.data.cache_store.local import LocalFileBackend, default_cache_root  # noqa: F40111