Skip to main content
RACS

RACS V1

Seeded robustness evaluation

RACS V1 evaluates whether observable operational degradation in an AMR-based warehouse can be detected early enough for a separate coordination layer to intervene before or during downstream propagation.

  • This is a discrete-event simulation.
  • This is not physical-robot validation.

Research question

Operational risk before hard failure

Can a separate risk-aware coordination layer use observable operational signals to intervene during progressive AMR degradation before or during broader system propagation?

System model

AMR transport feeding downstream work

  • Multiple AMRs transporting work
  • Task scheduler
  • Downstream workstation
  • Shared task arrivals
  • Progressive degradation of one AMR
  • RACS as a separate risk/coordination layer

V1 operational flow

  1. 01AMRs
  2. 02Transport tasks
  3. 03Workstation buffer
  4. 04Downstream processing

Experimental conditions

Three paired conditions

HEALTHY

Reference counterfactual

No degradation. Paired reference counterfactual.

REACTIVE BASELINE

Hard-failure recovery

Degrading robot remains available until hard failure. At hard failure, stranded work is requeued and reassigned.

RACS

Predictive coordination

Observable task behavior can trigger predictive drain before hard failure. The robot receives no new work while its existing task may complete. After drain completion it is cordoned. Hard-failure fallback remains available.

Observable-signal integrity

Observable evidence only

  • Localized anomaly is derived from task-age behavior relative to peers.
  • This supports suspect-robot anomaly attribution.
  • It is not root-cause diagnosis.

Seeded robustness design

Paired workloads across seed and WIP conditions

For each seed and WIP condition, healthy, reactive-baseline, and RACS runs use the same workload realization and shared non-treatment configuration.

Design parameterFrozen V1 value
Seeds30
Seed range1000 through 1029
WIP conditions0, 1, 2, 4
Conditionshealthy / reactive baseline / RACS
Total simulation runs360
Tasks per workloadexactly 51
Max arrivals per step2
Randomized factoronly arrival timing
Pairingsame arrival sequence for paired conditions
Automated tests248

Hypotheses

Frozen V1 outcomes

H1 — Transport coordination

PARTIALLY SUPPORTED

H2 — Earlier intervention and downstream benefit

NOT ESTABLISHED

H3 — Intervention headroom

SUPPORTED WITHIN TESTED WIP LEVELS

Metrics

Transport, downstream, and coordination measures

Metrics are grouped by analysis role; not all metrics showed improvement.

Transport

  • queue AUC
  • task latency
  • completed tasks
  • utilization

Downstream

  • workstation starvation / missed processing
  • paired excess missed processing
  • net downstream service deficit
  • downstream completion deficit
  • causal propagation timing

Coordination

  • intervention timing
  • intervention-before-propagation margin

Results

Frozen V1 findings

Lower queue AUC and lower latency are favorable.

Mean queue AUC delta, RACS − baseline

-2.9333

Mean latency delta, RACS − baseline

-0.0901

Queue AUC regressions

24 / 120 paired comparisons

Intervention-before-propagation fraction

WIP 0
66.7%
WIP 1
80.0%
WIP 2
93.3%
WIP 4
93.3%

Baseline causal cascade timing

WIPMinMedianMax
WIP 0162438
WIP 1162538
WIP 21728.545
WIP 4244459

Conditional analysis

H2 is not established because the intervention-before-propagation group did not show greater downstream benefit than the at/after group.

Group A

RACS intervention before baseline propagation

n = 100

net missed-processing delta
-0.130
downstream completion deficit delta
-0.130
positive excess delta
-0.180
queue AUC delta
-2.760
latency delta
-0.08692

Group B

intervention at/after propagation

n = 20

net missed-processing delta
-0.150
downstream completion deficit delta
-0.150
positive excess delta
-0.150
queue AUC delta
-3.800
latency delta
-0.1058

Experiment figures

Released V1 figures

Intervention before propagation by WIP

Higher disturbance-time WIP was associated with more frequent intervention before baseline propagation within the tested WIP levels.

Bar chart showing intervention-before-propagation fractions of 66.7 percent at WIP 0, 80.0 percent at WIP 1, and 93.3 percent at WIP 2 and WIP 4.
Fraction of paired RACS runs where intervention occurred before the paired reactive baseline's degradation-induced cascade start.

Queue AUC delta distribution

Mean queue AUC favored RACS, while 24 of 120 paired comparisons worsened.

Distribution plot of queue AUC deltas for RACS minus baseline, with some positive regression cases.
Paired queue AUC deltas use RACS minus reactive baseline; negative values favor RACS.

Latency delta distribution

Mean task latency favored RACS across the seeded study.

Distribution plot of average completion latency deltas for RACS minus baseline.
Paired latency deltas use RACS minus reactive baseline; negative values indicate lower completion latency under RACS.

Baseline cascade timing by WIP

Baseline propagation timing varied by seeded arrivals and shifted later at higher disturbance-time WIP.

Distribution plot showing baseline cascade start timing by WIP level, with later timing at higher WIP.
Reactive-baseline degradation-induced cascade start distributions for WIP 0, 1, 2, and 4.

Downstream net deficit delta distribution

Downstream deficit deltas did not establish H2's earlier-intervention benefit relationship.

Distribution plot of downstream net missed-processing deficit deltas for RACS minus baseline.
Paired net missed-processing deficit deltas use RACS minus reactive baseline; negative loss deltas favor RACS.

Representative timeline

The representative WIP 2 case shows the event sequence available in the frozen result artifact.

Timeline plot for the representative seed 1012 at WIP 2 showing degradation, intervention, and propagation events.
Representative case selected as the seed closest to median baseline queue AUC at WIP 2, breaking ties by lowest seed.

Negative cases

Retained negative results

  • RACS worsened transport queue AUC in 24 / 120 paired comparisons.
  • RACS worsened net downstream missed-processing deficit in 0 / 120 paired comparisons.
  • One observed cascade prevention occurred at WIP 4, but this is not a general result.

Interpretation

Scientific interpretation

No significance claims are made.

H1 — Transport coordination

PARTIALLY SUPPORTED

Partially supported. Mean transport metrics favored RACS, but not every paired realization improved.

H2 — Earlier intervention and downstream benefit

NOT ESTABLISHED

Not established. Earlier intervention did not produce greater downstream benefit in this V1 study.

H3 — Intervention headroom

SUPPORTED WITHIN TESTED WIP LEVELS

Supported within tested WIP levels. Higher disturbance-time WIP was associated with a greater fraction of runs where RACS intervened before propagation.

Limitations

What V1 does not establish

  • Discrete-event simulation
  • No physical-robot validation
  • Single site
  • One degrading AMR
  • Service-capacity degradation rather than full locomotion dynamics
  • Deterministic robot service duration
  • Abstract downstream workstation
  • Only workload-arrival timing randomized
  • 30 seeds
  • Descriptive statistics only
  • No statistical-significance claim
  • No ROS 2 / Gazebo validation
  • Downstream effect sizes remain small

Reproduction

Reproduce without overwriting reference artifacts

The reproduction creates a new result directory and does not overwrite the frozen reference artifacts.

Setup

git clone https://github.com/ridwanbale/racs-protocols.git
cd racs-protocols
python -m venv .venv
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -e ".[dev]"

Run

.\.venv\Scripts\python.exe -m simulations.experiments.racs_v1 `
  --seeds 30 `
  --first-seed 1000 `
  --output results `
  --experiment-id seeded_robustness_30_reproduction

Plot

.\.venv\Scripts\python.exe -m simulations.experiments.plot_racs_v1 `
  --result-dir results\seeded_robustness_30_reproduction `
  --output-dir docs\experiments\figures\racs_v1_reproduction

Reference RACS V1

Use the version and release identifiers below when referring to a specific evaluation of RACS V1.

Name
RACS V1.0 — Seeded Robustness Evaluation
Release
v1.0.0
Release date
September 2026
Release tip
0fc41a0
Frozen implementation snapshot
b4764da6193e013c3c2e1f86763597498bc6253f
License
Apache 2.0
Maintainer
Ridwan Bale
SourceReleaseExperiment reportReproduction