Upstream lobsters' own RSpec suite, run against a Roundhouse emit of the same checkout. Most failures name a Rails construct not yet modeled; some are answers that came out wrong, and a few are this harness's own gaps. That makes their tests the worklist generator — so the ranked causes below, not the pass count, are what this page is for.
4.0.9, the version their .ruby-version pins.
Current lobsters, not a snapshot. This tracks whatever upstream HEAD is and
records its SHA; it deliberately does not pin. A benchmark must freeze its input or its numbers stop
being comparable — but this answers a present-tense question, and pinning it would only manufacture
a second fossil beside the ruby-bench capture the
benchmark page measures.
Each run stands alone. There is no history here and no comparison to previous runs: numbers move when their code moves as well as when ours does, so a delta would attribute upstream's changes to us.
26 of 111 spec files in their suite are run here — spec/models. Request, feature, mailbox and routing specs are not run. Model specs exercise the compiler surface most densely; the request tier is next, and the capybara/mailbox tier is ledgered rather than attempted.
Occurrences churn; causes are the unit of work, so each carries a stable slug you can still refer to after another forty upstream commits. Owner says who can act: most of this is Roundhouse's own modeling debt.
Counts are how many examples are currently stopped by each cause, not how many will pass when it lands: an example is attributed to the first exception it raised, so clearing one cause can reveal another behind it.
| Blocked | Cause | Owner | |
|---|---|---|---|
| 35 | Ran, answered wrongexpectation-mismatch | roundhouse | |
No exception — the emitted code executed and produced a different answer than Rails does. Each needs individual diagnosis; they cluster by message no better than by coin flip, so this bucket is a count, not a cause. Keep it visible: it is where correctness bugs hide once the crash-shaped failures clear.
5 of 35 examples
| |||
| 32 | Symbol used where an Integer index is expectedsymbol-into-integer | roundhouse | |
Undiagnosed. Shape suggests a Hash emitted as an Array (or a symbol-keyed read lowered to an index) — needs a reduction before it can be ranked honestly.
5 of 32 examples
| |||
| 5 | errors is an Array, not an ActiveModel::Errorserrors-object-api | roundhouse | |
The runtime errors collection is Array[String]; attribute identity is lost at push, so errors[:attr] and errors.messages have nothing to answer with.
5 examples
| |||
| 1 | travel_to unavailable to specsharness-time-helpers | this harness | |
ActiveSupport::Testing::TimeHelpers is a spec-side helper stock Rails mixes into example groups. Ours to provide in the shim, not compiler debt.
1 example
| |||
| 1 | Nested constant not emittednested-constant-missing | roundhouse | |
Constants nested inside a model class (Hat::Mail, Notification::Notifiable) are dropped rather than emitted alongside the class.
1 example
| |||
| 69 | Not yet triagedunclassified | untriaged | |
No rule in bench/lobsters/spec-causes.json matches these. Triage, then add a rule.
10 of 69 examples
| |||
Rules matching nothing this run — either cleared, or the failure's message changed and is now counted as untriaged: relation-pick, model-class-first, update-attribute, apppath-concat, kernel-bigdecimal, nil-arith-coercion, dirty-saved-change-alias, has-attribute-predicate, scope-not-on-relation, binary-column-text-escape, searchparser-parslet-dsl, keystore-upsert.
| File | Examples | Pass | Fail |
|---|---|---|---|
spec/models/story_spec.rb | 81 | 47 | 34 |
spec/models/search_spec.rb | 33 | 9 | 24 |
spec/models/domain_spec.rb | 26 | 6 | 20 |
spec/models/link_spec.rb | 14 | 0 | 14 |
spec/models/user_spec.rb | 22 | 8 | 14 |
spec/models/comment_spec.rb | 25 | 18 | 7 |
spec/models/invitation_request_spec.rb | 7 | 3 | 4 |
spec/models/mod_note_spec.rb | 4 | 0 | 4 |
spec/models/invitation_spec.rb | 4 | 1 | 3 |
spec/models/moderation_spec.rb | 3 | 0 | 3 |
spec/models/notification_spec.rb | 16 | 13 | 3 |
spec/models/tag_spec.rb | 10 | 7 | 3 |
spec/models/comment_redirect_cache_spec.rb | 2 | 0 | 2 |
spec/models/message_spec.rb | 4 | 2 | 2 |
spec/models/vote_spec.rb | 13 | 11 | 2 |
spec/models/category_spec.rb | 6 | 5 | 1 |
spec/models/hat_spec.rb | 5 | 4 | 1 |
spec/models/mod_mail_spec.rb | 1 | 0 | 1 |
spec/models/story_image_spec.rb | 2 | 1 | 1 |
spec/models/comment_stat_spec.rb | 1 | 1 | 0 |
spec/models/hat_request_spec.rb | 3 | 3 | 0 |
spec/models/keystore_spec.rb | 4 | 4 | 0 |
spec/models/mastodon_app_spec.rb | 3 | 3 | 0 |
spec/models/search_parser_spec.rb | 78 | 78 | 0 |
spec/models/story_text_spec.rb | 1 | 1 | 0 |
spec/models/username_spec.rb | 1 | 1 | 0 |
The shim that boots the emit in place of Rails does these things on the emit's behalf. Listed because a pass rate achieved by propping up the subject is not a pass rate about the subject.
| Compensation | Kind | Why |
|---|---|---|
require "net/http" in the shim | emit gap | app/models/sponge.rb references Net:: constants but the emit never requires net/http. |
Numeric duration builders in the shim | by design | Specs and factories are untranspiled plain Ruby; the emit path rewrites n.days at transpile time, so the shared runtime never reopens Numeric. |
A spec file or hook that failed to run at all. These are not counted as failures above, so the denominator understates by however many examples they would have contributed.
An error occurred in an `after(:context)` hook.
Failure/Error: @stories.each(&:destroy!)
ActiveRecord::StatementInvalid:
no such column: links.story_id:
SELECT "links"."id" AS id, "links"."from_comment_id" AS from_comment_id, "links"."from_story_id" AS from_story_id, "links"."normalized_url" AS normalized_url, "links"."title" AS title, "links"."to_comment_id" AS to_comment_id, "links"."to_story_id" AS to_story_id, "links"."url" AS url FROM links WHERE (links."story_id" = ?)
^
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/db.rb:1278:in 'Db.raise_statement_error'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/db.rb:1347:in 'Db.prepare'
# /home/rubys/roundhouse/build/lobsters-specs/emit/app/models/link.rb:527:in 'Link._hydrate_all'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/relation.rb:1535:in 'ActiveRecord::Relation#load_records'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/relation.rb:1694:in 'ActiveRecord::Relation#loaded_records'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/relation.rb:1701:in 'ActiveRecord::Relation#each'
# /home/rubys/roundhouse/build/lobsters-specs/emit/app/models/story.rb:2759:in 'Story#before_destroy'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1899:in 'block (2 levels) in ActiveRecord::Base#destroy'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1918:in 'block in ActiveRecord::Base#_halted_by_abort'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1917:in 'Kernel#catch'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1917:in 'ActiveRecord::Base#_halted_by_abort'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1899:in 'block in ActiveRecord::Base#destroy'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:936:in 'ActiveRecord::Base.transaction'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1898:in 'ActiveRecord::Base#destroy'
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/active_record/connection.rb:1911:in 'ActiveRecord::Base#destroy!'
# /home/rubys/lobsters/spec/models/search_spec.rb:62:in 'Array#each'
# /home/rubys/lobsters/spec/models/search_spec.rb:62:in 'block (2 levels) in <top (required)>'
# ------------------
# --- Caused by: ---
# SQLite3::SQLException:
# no such column: links.story_id:
# SELECT "links"."id" AS id, "links"."from_comment_id" AS from_comment_id, "links"."from_story_id" AS from_story_id, "links"."normalized_url" AS normalized_url, "links"."title" AS title, "links"."to_comment_id" AS to_comment_id, "links"."to_story_id" AS to_story_id, "links"."url" AS url FROM links WHERE (links."story_id" = ?)
# ^
# /home/rubys/roundhouse/build/lobsters-specs/emit/runtime/db.rb:1387:in 'Db.prepare_plain'
From the transpile that produced the tree under test: 1,085 files, 0 errors, 2,009 warnings. Warnings are a modeling-debt ledger rather than a failure — the Ruby target emits through its gaps — so they are not comparable to the spec failures above.
| Diagnostic | Count |
|---|---|
gradual_untyped | 1,043 |
unresolved_type | 457 |
lower_residue | 380 |
unconsumed_initializer | 52 |
unsupported | 32 |
incompatible_binop | 15 |
missing_preload | 14 |
send_dispatch_failed | 9 |
ivar_unresolved | 7 |