{"id":14880,"date":"2026-10-02T17:30:00","date_gmt":"2026-10-02T12:00:00","guid":{"rendered":"https:\/\/www.allerin.com\/blog\/?p=14880"},"modified":"2026-09-25T11:20:51","modified_gmt":"2026-09-25T05:50:51","slug":"rails-upgrade-in-ci","status":"publish","type":"post","link":"https:\/\/www.allerin.com\/blog\/rails-upgrade-in-ci\/","title":{"rendered":"Testing a Rails upgrade in CI without hiding failures"},"content":{"rendered":"<p>A Rails upgrade in CI needs a passing current-version job and a visible result for the target version. Keep main shipping, run both dependency graphs, reject new deprecations and reproduce failures before changing the framework version.<\/p>\n<p>An experimental job may fail without failing the workflow. That is useful while removing upgrade blockers, but the green workflow badge cannot approve the final switch. Remove the target job\u2019s failure allowance and require its check before merging that switch.<\/p>\n<nav style=\"margin: 24px 0; padding: 18px; background: #edf4f0; border-radius: 6px;\" aria-label=\"Article sections\">\n<p style=\"margin: 0 0 12px;\"><strong>Find the relevant check<\/strong><\/p>\n<ul style=\"display: flex; flex-wrap: wrap; gap: 10px 24px; list-style: none; margin: 0; padding: 0;\">\n<li style=\"margin: 0;\"><a href=\"#give-each-rails-version-a-visible-result\">Give each Rails version a visible result<\/a><\/li>\n<li style=\"margin: 0;\"><a href=\"#run-the-same-checks-locally\">Run the same checks locally<\/a><\/li>\n<li style=\"margin: 0;\"><a href=\"#reject-new-warnings-without-hiding-known-ones\">Reject new warnings without hiding known ones<\/a><\/li>\n<li style=\"margin: 0;\"><a href=\"#check-what-the-environment-can-conceal\">Check what the environment can conceal<\/a><\/li>\n<li style=\"margin: 0;\"><a href=\"#reproduce-a-flaky-failure-before-quarantining-it\">Reproduce a flaky failure before quarantining it<\/a><\/li>\n<li style=\"margin: 0;\"><a href=\"#what-to-do-this-week\">What to do this week<\/a><\/li>\n<\/ul>\n<\/nav>\n<h2 id=\"give-each-rails-version-a-visible-result\" style=\"scroll-margin-top: 128px;\">Give each Rails version a visible result<\/h2>\n<p>The generated application accompanying this article keeps Ruby 3.4.10 and Rails defaults at 8.0 while testing Rails 8.0.5.1 and 8.1.3.1 against PostgreSQL 15. Separate lockfiles preserve each dependency graph. These are fixture choices, not a claim about Allerin production infrastructure.<\/p>\n<p><a href=\"https:\/\/github.com\/fastruby\/next_rails\/blob\/v1.7.0\/README.md\" target=\"_blank\" rel=\"noopener\">next_rails 1.7.0<\/a> supplies the switch. Its <code style=\"overflow-wrap: anywhere; white-space: normal;\">--init<\/code> command creates <code style=\"overflow-wrap: anywhere; white-space: normal;\">Gemfile.next<\/code> and a <code style=\"overflow-wrap: anywhere; white-space: normal;\">next?<\/code> helper for the Gemfile. The application helper is <code style=\"overflow-wrap: anywhere; white-space: normal;\">NextRails.next?<\/code>. The recorded setup uses:<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-sh\">bundle exec next_rails --init\r\nbundle exec next bundle install\r\nBUNDLE_GEMFILE=Gemfile.next bin\/rails test\r\n<\/code><\/pre>\n<p>Use dependencies that support both frameworks, merge compatible repairs into main, and keep the framework switch small. The fixture\u2019s first form-request test caught a dependency failure before that switch. The current-version bundle resolved JSON 3.0.0, then raised <code style=\"overflow-wrap: anywhere; white-space: normal;\">ArgumentError: unknown keyword: quirks_mode<\/code>. Constraining the shared JSON dependency to the compatible 2.x line preserved the application assertion. JSON 3.0.0, released on 7 September 2026, tightened option validation; that change and the exact Rails encoder explain this recorded path. With JSON 2.21.2, both serial suites passed seven tests and 18 assertions. <a href=\"https:\/\/github.com\/ruby\/json\/blob\/v3.0.0\/CHANGES.md\" target=\"_blank\" rel=\"noopener\">JSON change<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.0.5.1\/activesupport\/lib\/active_support\/json\/encoding.rb#L110\" target=\"_blank\" rel=\"noopener\">Rails 8.0 encoder<\/a>.<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-yaml\">name: Rails upgrade\r\non: [push, pull_request]\r\npermissions:\r\n  contents: read\r\nconcurrency:\r\n  group: ${{ github.workflow }}-${{ github.ref }}\r\n  cancel-in-progress: true\r\njobs:\r\n  test:\r\n    name: Rails (${{ matrix.gemfile }})\r\n    runs-on: ubuntu-24.04\r\n    timeout-minutes: 15\r\n    continue-on-error: ${{ matrix.experimental }}\r\n    strategy:\r\n      fail-fast: false\r\n      matrix:\r\n        include:\r\n          - gemfile: Gemfile\r\n            experimental: false\r\n            timezone: UTC\r\n          - gemfile: Gemfile.next\r\n            experimental: true\r\n            timezone: America\/New_York\r\n    services:\r\n      postgres:\r\n        image: postgres:15.17\r\n        env:\r\n          POSTGRES_PASSWORD: fixture-only\r\n        ports: ['5432:5432']\r\n        options: &gt;-\r\n          --health-cmd pg_isready --health-interval 10s\r\n          --health-timeout 5s --health-retries 5\r\n    env:\r\n      BUNDLE_GEMFILE: ${{ matrix.gemfile }}\r\n      DATABASE_URL: postgres:\/\/postgres:fixture-only@localhost:5432\/allerin_ci67_test\r\n      RAILS_ENV: test\r\n      CI: '1'\r\n      SECRET_KEY_BASE: public-ci-fixture-only-not-a-deployment-secret-public-ci-fixture-only\r\n      RUBYOPT: '-W:deprecated'\r\n      PARALLEL_WORKERS: '2'\r\n      TZ: ${{ matrix.timezone }}\r\n    steps:\r\n      - uses: actions\/checkout@v4\r\n      - uses: ruby\/setup-ruby@v1\r\n        with:\r\n          ruby-version: '3.4.10'\r\n          bundler: '2.5.22'\r\n          bundler-cache: true\r\n      - run: bin\/rails db:prepare\r\n      - name: Run every check\r\n        run: |\r\n          if [ \"$BUNDLE_GEMFILE\" = Gemfile.next ]; then\r\n            bin\/ci\r\n          else\r\n            bundle exec ruby script\/ci_legacy.rb\r\n          fi\r\n      - uses: actions\/upload-artifact@v4\r\n        if: always()\r\n        with:\r\n          name: rails-${{ matrix.gemfile }}\r\n          path: |\r\n            log\/test.log\r\n            tmp\/screenshots\/\r\n          retention-days: 7\r\n<\/code><\/pre>\n<p>This workflow deliberately permits the target job to fail while the upgrade is unfinished. Inspect that job\u2019s steps and retained output separately. <code style=\"overflow-wrap: anywhere; white-space: normal;\">fail-fast: false<\/code> lets both cells report. The concurrency group cancels superseded runs, and the timeout limits a stuck job. These controls bound waste; they do not promise that two cells cost exactly twice as much. <a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/workflows-and-actions\/workflow-syntax\" target=\"_blank\" rel=\"noopener\">Workflow semantics<\/a>.<\/p>\n<p>Rails 7.2 introduced generated GitHub CI, Brakeman and the RuboCop omakase configuration in 2024. The released 8.1.3.1 workflow still lacks the 15-minute timeout added to <code style=\"overflow-wrap: anywhere; white-space: normal;\">main<\/code> by PR #58595 on 28 August 2026. The example adopts that limit explicitly. <a href=\"https:\/\/guides.rubyonrails.org\/7_2_release_notes.html\" target=\"_blank\" rel=\"noopener\">7.2 release<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/pull\/58595\" target=\"_blank\" rel=\"noopener\">timeout change<\/a>.<\/p>\n<h2 id=\"run-the-same-checks-locally\" style=\"scroll-margin-top: 128px;\">Run the same checks locally<\/h2>\n<p>Rails 8.1 introduced <code style=\"overflow-wrap: anywhere; white-space: normal;\">bin\/ci<\/code> and <code style=\"overflow-wrap: anywhere; white-space: normal;\">config\/ci.rb<\/code> in 2025. Its runner executes all steps, reports their results and exits unsuccessfully if any failed. It does not stop after the first failure. Here is the complete fixture definition. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.3.1\/activesupport\/lib\/active_support\/continuous_integration.rb\" target=\"_blank\" rel=\"noopener\">Runner source<\/a>.<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-ruby\">CI.run do\r\n  step \"Style\", \"bundle exec rubocop\"\r\n  step \"Security scan\", \"bundle exec brakeman --no-pager\"\r\n  step \"Gem audit\", \"bundle exec bundler-audit check --update\"\r\n  step \"JavaScript audit\", \"bin\/importmap audit\"\r\n  step \"Autoload\", \"bin\/rails zeitwerk:check\"\r\n  step \"Schema drift\", \"sh script\/check_schema.sh\"\r\n  step \"Unit and integration tests\", \"bin\/rails test\"\r\n  step \"System tests\", \"bin\/rails test:system\"\r\nend\r\n<\/code><\/pre>\n<p>The current Rails 8.0 cell cannot load the 8.1 runner. Its equivalent plain script retains the same checks and final failure status:<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-ruby\">require \"rbconfig\"\r\n$stdout.sync = true\r\nENV[\"CI\"] = \"true\"\r\nENV[\"PATH\"] = \"#{File.dirname(RbConfig.ruby)}#{File::PATH_SEPARATOR}#{ENV.fetch('PATH')}\"\r\nsteps = [\r\n  [ \"Style\", \"bundle exec rubocop\" ],\r\n  [ \"Security scan\", \"bundle exec brakeman --no-pager\" ],\r\n  [ \"Gem audit\", \"bundle exec bundler-audit check --update\" ],\r\n  [ \"JavaScript audit\", \"bin\/importmap audit\" ],\r\n  [ \"Autoload\", \"bin\/rails zeitwerk:check\" ],\r\n  [ \"Schema drift\", \"sh script\/check_schema.sh\" ],\r\n  [ \"Unit and integration tests\", \"bin\/rails test\" ],\r\n  [ \"System tests\", \"bin\/rails test:system\" ]\r\n]\r\nresults = steps.map do |label, command|\r\n  puts \"\\n#{label}\"\r\n  success = system(command)\r\n  [ label, success ]\r\nend\r\nabort \"Failed steps: #{results.reject(&amp;:last).map(&amp;:first).join(', ')}\" unless results.all?(&amp;:last)\r\n<\/code><\/pre>\n<p>The lockfiles identify Brakeman 8.0.6, bundler-audit 0.9.3, rubocop-rails-omakase 1.1.0 and importmap-rails 2.2.3. The advisory database is refreshed by the audit command. <code style=\"overflow-wrap: anywhere; white-space: normal;\">bin\/importmap audit<\/code> applies to this importmap application; a different JavaScript package system needs its own check. Passing scans describe their findings, not a security certification.<\/p>\n<p>Both Linux cells completed all eight pipeline steps. Each ran seven tests and 18 assertions in two processes, plus one headless Chrome test with two assertions. The validation wrapper also checked deliberate failures and required both cells to pass. A local macOS forked database connection crashed before an assertion result; local serial runs and Linux process runs are recorded separately. <a href=\"https:\/\/www.allerin.com\/rails-checks\/rails-upgrade-ci-2026-10-results.json\">Recorded results<\/a>.<\/p>\n<p>A 2007 Rails core discussion about CruiseControl.rb asked for the breaking changeset and useful failure details. Preserve that information in today\u2019s artifacts too. A link to an empty report is still unhelpful. <a href=\"https:\/\/discuss.rubyonrails.org\/t\/rubyonrails-build-6553-failed\/12055\" target=\"_blank\" rel=\"noopener\">Original discussion<\/a>.<\/p>\n<h2 id=\"reject-new-warnings-without-hiding-known-ones\" style=\"scroll-margin-top: 128px;\">Reject new warnings without hiding known ones<\/h2>\n<p>Rails 6.1 added disallowed-deprecation controls in 2020, before the current time-based maintenance policy. <code style=\"overflow-wrap: anywhere; white-space: normal;\">config.active_support.disallowed_deprecation_warnings<\/code> selects warnings for <code style=\"overflow-wrap: anywhere; white-space: normal;\">config.active_support.disallowed_deprecation<\/code>. It is a denylist. A growing list can prevent repaired warnings returning, but it cannot catch every new warning. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v6.1.0\/activesupport\/lib\/active_support\/deprecation\/disallowed.rb\" target=\"_blank\" rel=\"noopener\">Disallowed warnings<\/a>.<\/p>\n<p>The fixture uses a narrow known-warning allowlist and raises for everything else:<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-ruby\">module CiDeprecationPolicy\r\n  KNOWN = [ \"CI fixture old API will be removed\" ].freeze\r\n  HANDLER = lambda do |message, _callstack|\r\n    normalized = message.sub(\/\\ADEPRECATION WARNING: \/, \"\").sub(\/ \\(called from .*\\z\/m, \"\")\r\n    if KNOWN.include?(normalized)\r\n      warn \"Known deprecation: #{normalized}\"\r\n    else\r\n      raise ActiveSupport::DeprecationException, message\r\n    end\r\n  end\r\nend\r\n<\/code><\/pre>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-ruby\"># config\/environments\/test.rb, Rails 7.1 and later\r\nrequire_relative \"..\/..\/lib\/ci_deprecation_policy\"\r\nRails.application.configure do\r\n  config.active_support.deprecation = CiDeprecationPolicy::HANDLER\r\nend\r\n\r\n<\/code><\/pre>\n<p>For the separately exercised Active Support 6.1.7.10 probe, load the same policy and use the older setter:<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-ruby\">ActiveSupport::Deprecation.behavior = CiDeprecationPolicy::HANDLER\r\n<\/code><\/pre>\n<p>Rails 7.1\u2019s 2023 per-library deprecators changed the object being configured. <code style=\"overflow-wrap: anywhere; white-space: normal;\">Rails.application.deprecators<\/code> manages registered deprecators, including later additions. The separate 6.1 probe uses the older singleton setter. Both probes permit the known warning and reject the unknown one with <code style=\"overflow-wrap: anywhere; white-space: normal;\">ActiveSupport::DeprecationException<\/code>. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.1.0\/activesupport\/lib\/active_support\/deprecation\/deprecators.rb\" target=\"_blank\" rel=\"noopener\">Collection<\/a>.<\/p>\n<p>For a large existing backlog, deprecation_toolkit 2.4.0 can maintain a reviewed per-test baseline. Its record mode passed, the unchanged baseline passed, and controlled additions and removals failed with <code style=\"overflow-wrap: anywhere; white-space: normal;\">DeprecationIntroduced<\/code> and <code style=\"overflow-wrap: anywhere; white-space: normal;\">DeprecationRemoved<\/code>. Remove obsolete exceptions and review every addition so the backlog shrinks. Recording belongs in a deliberate baseline change, never an unconditional CI step. <a href=\"https:\/\/github.com\/Shopify\/deprecation_toolkit\/blob\/v2.4.0\/README.md\" target=\"_blank\" rel=\"noopener\">Toolkit instructions<\/a>.<\/p>\n<p>Ruby 3.4\u2019s 2024 release also changed <code style=\"overflow-wrap: anywhere; white-space: normal;\">Hash#inspect<\/code> spacing. Compare values when formatting is irrelevant; keep explicit output assertions where presentation is the feature. The <a href=\"https:\/\/www.allerin.com\/blog\/ruby-3-3-to-3-4-upgrade\/\">Ruby upgrade reproduction<\/a> records that failure separately. <a href=\"https:\/\/www.ruby-lang.org\/en\/news\/2024\/12\/25\/ruby-3-4-0-released\/\" target=\"_blank\" rel=\"noopener\">Ruby release<\/a>.<\/p>\n<p><code style=\"overflow-wrap: anywhere; white-space: normal;\">RUBYOPT=\"-W:deprecated\"<\/code> also reveals Ruby warnings, including Ruby 3.4\u2019s chilled-string warnings. It does not itself turn those warnings into build failures. A Rails deprecation handler does not automatically govern Ruby\u2019s warning channel.<\/p>\n<h2 id=\"check-what-the-environment-can-conceal\" style=\"scroll-margin-top: 128px;\">Check what the environment can conceal<\/h2>\n<p>Rails 7.2 changed tests to respect an explicitly configured Active Job adapter consistently. Without configuration, the test adapter remains the fallback. An application using Solid Queue must still decide whether a test checks enqueue intent or a real worker. <a href=\"https:\/\/guides.rubyonrails.org\/v7.2\/upgrading_ruby_on_rails.html#all-tests-now-respect-the-active-job-queue-adapter-config\" target=\"_blank\" rel=\"noopener\">7.2 upgrade guide<\/a>.<\/p>\n<p>In both tested versions, asserting that <code style=\"overflow-wrap: anywhere; white-space: normal;\">perform_later<\/code> had already changed a record failed. Asserting enqueue intent passed; wrapping the enqueue in <code style=\"overflow-wrap: anywhere; white-space: normal;\">perform_enqueued_jobs<\/code> passed when the assertion needed the performed result. An explicitly configured inline adapter also made <code style=\"overflow-wrap: anywhere; white-space: normal;\">assert_enqueued_with<\/code> reject the setup because that helper requires the test adapter.<\/p>\n<p>When checking error pages, inspect Rails 7.1\u2019s 2023 <code style=\"overflow-wrap: anywhere; white-space: normal;\">show_exceptions<\/code> values and your application\u2019s chosen setting. The generated <code style=\"overflow-wrap: anywhere; white-space: normal;\">:rescuable<\/code> setting renders configured HTTP errors while other exceptions still raise. <a href=\"https:\/\/guides.rubyonrails.org\/upgrading_ruby_on_rails.html#new-ways-to-handle-exceptions-in-controller-tests-integration-tests-and-system-tests\" target=\"_blank\" rel=\"noopener\">Upgrade guide<\/a>.<\/p>\n<p>Strict loading, introduced in Rails 6.1 in 2020, can expose unintended association loads; adopt it deliberately where the suite can assert the expected queries. <a href=\"https:\/\/guides.rubyonrails.org\/6_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">6.1 release<\/a>.<\/p>\n<p>Use the production database engine and major version in CI. SQLite is not a substitute for testing PostgreSQL types, constraints or query behavior. Keep <code style=\"overflow-wrap: anywhere; white-space: normal;\">RAILS_ENV=test<\/code> explicit for the autoload and migration checks:<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-sh\">set -eu\r\nexport RAILS_ENV=test\r\nbin\/rails zeitwerk:check\r\ngit ls-files --error-unmatch db\/schema.rb db\/schema.next.rb &gt;\/dev\/null\r\nbin\/rails db:migrate\r\ngit diff --exit-code -- db\/schema.rb db\/schema.next.rb\r\n<\/code><\/pre>\n<p>The shared schema baseline failed the next-version drift check. Rails 8.1 changed its version header and column order. The repair keeps reviewed <code style=\"overflow-wrap: anywhere; white-space: normal;\">schema.rb<\/code> and <code style=\"overflow-wrap: anywhere; white-space: normal;\">schema.next.rb<\/code> baselines, selected through <code style=\"overflow-wrap: anywhere; white-space: normal;\">database.yml<\/code>\u2019s <code style=\"overflow-wrap: anywhere; white-space: normal;\">schema_dump<\/code> setting. Both checks then passed. The gate still rejects an intentional edit. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.0\/activerecord\/CHANGELOG.md\" target=\"_blank\" rel=\"noopener\">8.1 schema change<\/a>.<\/p>\n<p>The pipeline does not establish safe lock duration on production tables. Review migration operations separately; strong_migrations 2.8.0 is an optional check, not an executed part of this fixture. <a href=\"https:\/\/github.com\/ankane\/strong_migrations\/blob\/v2.8.0\/README.md\" target=\"_blank\" rel=\"noopener\">Gem source<\/a>.<\/p>\n<p>Rails 7.0\u2019s 2021 generator introduced <code style=\"overflow-wrap: anywhere; white-space: normal;\">config.eager_load = ENV[\"CI\"].present?<\/code>. Upgrading gems does not rewrite an older application\u2019s <code style=\"overflow-wrap: anywhere; white-space: normal;\">test.rb<\/code>. <code style=\"overflow-wrap: anywhere; white-space: normal;\">zeitwerk:check<\/code>, available with Rails 6.0\u2019s 2019 autoloader, provides a named check rather than relying on which test happens to load a constant. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v6.0.0\/railties\/lib\/rails\/tasks\/zeitwerk.rake\" target=\"_blank\" rel=\"noopener\">6.0 task<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.0.0\/railties\/lib\/rails\/generators\/rails\/app\/templates\/config\/environments\/test.rb.tt\" target=\"_blank\" rel=\"noopener\">Test environment<\/a>.<\/p>\n<h2 id=\"reproduce-a-flaky-failure-before-quarantining-it\" style=\"scroll-margin-top: 128px;\">Reproduce a flaky failure before quarantining it<\/h2>\n<p>Rails 4.0 moved from Test::Unit to MiniTest in 2013; 4.1 adopted <code style=\"overflow-wrap: anywhere; white-space: normal;\">Minitest::Test<\/code> in 2014. Rails 5.0 made random ordering the ActiveSupport test default in 2016. A seed helps reproduce a run; it does not repair shared state. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v4.0.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">4.0 source<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v4.1.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">4.1 source<\/a>, <a href=\"https:\/\/guides.rubyonrails.org\/upgrading_ruby_on_rails.html\" target=\"_blank\" rel=\"noopener\">upgrade guide<\/a>.<\/p>\n<pre style=\"overflow-x: auto; max-width: 100%;\"><code class=\"language-sh\">bin\/rails test checks\/order_dependence_test.rb --seed 3911\r\nbundle exec minitest_bisect --seed 3911 -Itest checks\/order_dependence_test.rb\r\nbin\/rails test checks\/order_dependence_fixed_test.rb --seed 3911\r\n<\/code><\/pre>\n<p>The recorded failure expected <code style=\"overflow-wrap: anywhere; white-space: normal;\">UTC<\/code> and received <code style=\"overflow-wrap: anywhere; white-space: normal;\">Hawaii<\/code>. minitest-bisect 1.8.0 reduced four tests to the two that leaked and observed the application\u2019s zone. Restoring the zone after each test made the same seed pass four assertions. Process <code style=\"overflow-wrap: anywhere; white-space: normal;\">TZ<\/code> and Rails\u2019 application time zone are separate settings. <a href=\"https:\/\/github.com\/seattlerb\/minitest-bisect\/blob\/v1.8.0\/README.rdoc\" target=\"_blank\" rel=\"noopener\">Bisect instructions<\/a>.<\/p>\n<p>Rails 6.0 added parallel tests in 2019. Rails 7.0 added the default threshold of 50 tests; the normal condition is more than 50, not 50 or more. Setting <code style=\"overflow-wrap: anywhere; white-space: normal;\">PARALLEL_WORKERS<\/code> can override that threshold. Treat CPU count as a starting budget and account for database connections and memory. The fixture forces two workers on Linux. <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.0.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">7.0 threshold<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.3.1\/activesupport\/lib\/active_support\/testing\/parallelize_executor.rb\" target=\"_blank\" rel=\"noopener\">Threshold source<\/a>.<\/p>\n<div style=\"overflow-x: auto;\" tabindex=\"0\" role=\"region\" aria-label=\"Flaky test causes and the checks that expose them\">\n<table style=\"width: 100%; min-width: 620px; table-layout: auto; border-collapse: collapse; font-size: 16px; line-height: 1.55;\" aria-label=\"Flaky test causes and the checks that expose them\">\n<thead>\n<tr>\n<th style=\"padding: 14px; border: 1px solid #d3dfdb; background: #edf4f0; text-align: left; vertical-align: top;\" scope=\"col\">Failure cause<\/th>\n<th style=\"padding: 14px; border: 1px solid #d3dfdb; background: #edf4f0; text-align: left; vertical-align: top;\" scope=\"col\">Check and repair<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Order or shared configuration<\/td>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Reproduce the seed, bisect the tests and restore state after each example.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Time and implicit time zones<\/td>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Use <code style=\"overflow-wrap: anywhere; white-space: normal;\">travel_to<\/code> or <code style=\"overflow-wrap: anywhere; white-space: normal;\">freeze_time<\/code>; run a non-UTC cell and restore the application zone.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">External network<\/td>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Stub expected calls and reject unexpected access instead of accepting intermittent service behavior.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Browser and database state<\/td>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Check transaction visibility and worker connections. Rails 5.1\u2019s 2017 shared test connections do not cover every separate-process or multiple-database arrangement.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Random input<\/td>\n<td style=\"padding: 14px; border: 1px solid #d3dfdb; text-align: left; vertical-align: top; overflow-wrap: anywhere;\">Retain the seed and failing input; replace accidental assumptions with a stated invariant.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><code style=\"overflow-wrap: anywhere; white-space: normal;\">travel_to<\/code> arrived in Rails 4.1 in 2014; <code style=\"overflow-wrap: anywhere; white-space: normal;\">freeze_time<\/code> in Rails 5.2 in 2018. System tests and shared transactional test connections arrived in 5.1. Quarantine a remaining flaky test only with an owner and linked issue. Blanket retries hide evidence. <a href=\"https:\/\/guides.rubyonrails.org\/4_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">4.1 release<\/a>, <a href=\"https:\/\/github.com\/rails\/rails\/blob\/v5.2.0\/activesupport\/lib\/active_support\/testing\/time_helpers.rb\" target=\"_blank\" rel=\"noopener\">Time helpers<\/a>, <a href=\"https:\/\/guides.rubyonrails.org\/5_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">system-test release<\/a>.<\/p>\n<p>The unordered-select shuffle in PR #58548 was on <code style=\"overflow-wrap: anywhere; white-space: normal;\">main<\/code> at the 7 September check, disabled by default. It does not cover every SQL path and is not a released setting in the tested frameworks. No Rails 8.2 delivery date is assumed. <a href=\"https:\/\/github.com\/rails\/rails\/pull\/58548\" target=\"_blank\" rel=\"noopener\">Shuffle discussion<\/a>.<\/p>\n<h2 id=\"what-to-do-this-week\" style=\"scroll-margin-top: 128px;\">What to do this week<\/h2>\n<ol>\n<li>Establish passing current-version checks, then add the target lockfile and visible experimental job.<\/li>\n<li>Turn a demonstrated failure into a check, review the warning baseline and merge compatible repairs on main.<\/li>\n<li>Match the database engine and major, exercise browser and job behavior, and retain seeds, logs and screenshots.<\/li>\n<li>Remove the target job\u2019s failure allowance and require its check before the framework switch. Rehearse deployment separately; CI does not prove production migration safety.<\/li>\n<\/ol>\n<p>FastRuby\u2019s September article demonstrates the same principle for website audits, with content, link and metadata checks. Its scope is distinct from the upgrade experiments here. <a href=\"https:\/\/www.fastruby.io\/blog\/turning-audit-findings-into-ci-checks.html\" target=\"_blank\" rel=\"noopener\">Audit checks<\/a>.<\/p>\n<p><strong>Methods checked 7 September 2026.<\/strong> Codex researched tagged sources and executed a generated application and deliberate failures. These are bounded reproductions, not a client case study or byline-owner production account. <a href=\"https:\/\/www.allerin.com\/rails-checks\/rails-upgrade-ci-2026-10.zip\">Download the application, lockfiles, workflow and verification script<\/a>. The samples are not published as a separate public repository. Retest the recorded dependencies against current patches before an application upgrade.<\/p>\n<p>For the application\u2019s upgrade sequence, use Allerin\u2019s <a href=\"https:\/\/www.allerin.com\/services\/rails-upgrades\">Rails upgrade guide<\/a>.<\/p>\n<details>\n<summary>Sources checked on 7 September 2026<\/summary>\n<ol>\n<li><a href=\"https:\/\/github.com\/fastruby\/next_rails\/blob\/v1.7.0\/README.md\" target=\"_blank\" rel=\"noopener\">next_rails 1.7.0<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ruby\/json\/blob\/v3.0.0\/CHANGES.md\" target=\"_blank\" rel=\"noopener\">JSON change<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.0.5.1\/activesupport\/lib\/active_support\/json\/encoding.rb#L110\" target=\"_blank\" rel=\"noopener\">Rails 8.0 encoder<\/a><\/li>\n<li><a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/workflows-and-actions\/workflow-syntax\" target=\"_blank\" rel=\"noopener\">Workflow semantics<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/7_2_release_notes.html\" target=\"_blank\" rel=\"noopener\">7.2 release<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/pull\/58595\" target=\"_blank\" rel=\"noopener\">timeout change<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.3.1\/activesupport\/lib\/active_support\/continuous_integration.rb\" target=\"_blank\" rel=\"noopener\">Runner source<\/a><\/li>\n<li><a href=\"https:\/\/discuss.rubyonrails.org\/t\/rubyonrails-build-6553-failed\/12055\" target=\"_blank\" rel=\"noopener\">Original discussion<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v6.1.0\/activesupport\/lib\/active_support\/deprecation\/disallowed.rb\" target=\"_blank\" rel=\"noopener\">Disallowed warnings<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.1.0\/activesupport\/lib\/active_support\/deprecation\/deprecators.rb\" target=\"_blank\" rel=\"noopener\">Collection<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/Shopify\/deprecation_toolkit\/blob\/v2.4.0\/README.md\" target=\"_blank\" rel=\"noopener\">Toolkit instructions<\/a><\/li>\n<li><a href=\"https:\/\/www.ruby-lang.org\/en\/news\/2024\/12\/25\/ruby-3-4-0-released\/\" target=\"_blank\" rel=\"noopener\">Ruby release<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/v7.2\/upgrading_ruby_on_rails.html#all-tests-now-respect-the-active-job-queue-adapter-config\" target=\"_blank\" rel=\"noopener\">7.2 upgrade guide<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/upgrading_ruby_on_rails.html#new-ways-to-handle-exceptions-in-controller-tests-integration-tests-and-system-tests\" target=\"_blank\" rel=\"noopener\">Upgrade guide<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/6_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">6.1 release<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.0\/activerecord\/CHANGELOG.md\" target=\"_blank\" rel=\"noopener\">8.1 schema change<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ankane\/strong_migrations\/blob\/v2.8.0\/README.md\" target=\"_blank\" rel=\"noopener\">Gem source<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v6.0.0\/railties\/lib\/rails\/tasks\/zeitwerk.rake\" target=\"_blank\" rel=\"noopener\">6.0 task<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.0.0\/railties\/lib\/rails\/generators\/rails\/app\/templates\/config\/environments\/test.rb.tt\" target=\"_blank\" rel=\"noopener\">Test environment<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v4.0.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">4.0 source<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v4.1.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">4.1 source<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/upgrading_ruby_on_rails.html\" target=\"_blank\" rel=\"noopener\">upgrade guide<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/seattlerb\/minitest-bisect\/blob\/v1.8.0\/README.rdoc\" target=\"_blank\" rel=\"noopener\">Bisect instructions<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v7.0.0\/activesupport\/lib\/active_support\/test_case.rb\" target=\"_blank\" rel=\"noopener\">7.0 threshold<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v8.1.3.1\/activesupport\/lib\/active_support\/testing\/parallelize_executor.rb\" target=\"_blank\" rel=\"noopener\">Threshold source<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/4_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">4.1 release<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/blob\/v5.2.0\/activesupport\/lib\/active_support\/testing\/time_helpers.rb\" target=\"_blank\" rel=\"noopener\">Time helpers<\/a><\/li>\n<li><a href=\"https:\/\/guides.rubyonrails.org\/5_1_release_notes.html\" target=\"_blank\" rel=\"noopener\">system-test release<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rails\/rails\/pull\/58548\" target=\"_blank\" rel=\"noopener\">Shuffle discussion<\/a><\/li>\n<li><a href=\"https:\/\/www.fastruby.io\/blog\/turning-audit-findings-into-ci-checks.html\" target=\"_blank\" rel=\"noopener\">Audit checks<\/a><\/li>\n<\/ol>\n<\/details>\n","protected":false},"excerpt":{"rendered":"<p>A runnable Rails CI fixture exposes dependency mismatches, misleading job assertions, schema drift and shared test state, with failures and repairs recorded.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[2037],"tags":[2062,2060,2063,2061,2038],"class_list":["post-14880","post","type-post","status-publish","format-standard","hentry","category-ruby-on-rails","tag-active-job","tag-continuous-integration","tag-minitest","tag-rails-testing","tag-rails-upgrades"],"_links":{"self":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/comments?post=14880"}],"version-history":[{"count":2,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14880\/revisions"}],"predecessor-version":[{"id":14895,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14880\/revisions\/14895"}],"wp:attachment":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/media?parent=14880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/categories?post=14880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/tags?post=14880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}