Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Jan 15, 2025

Infrastructure

Gateway​

• Added beacon API RPC endpoints for Ethereum based on Lighthouse
  • More information on Telegram
  • Next step: add Nimbus

Innovation

Shutter​

• Call with Marc to debug libp2p issues

• Working on devtooling to make testing easier

• Not present next week

Testing

Hive​

• No major update

Ethereum/tests​

• Working on bug fixes for Pectra

Client updates

EL​

Nethermind​

• Got 2 reports about missing receipts on archive nodes
  • Potentially found the issue
  • A recovery method could be possible
  • Quite a big edge case

Erigon​

• Shutter
  • Basic p2p connection working for Shutter
  • Will join the Shutter libp2p call
  • Working on reading data from the Shutter smart contract needed for sequencing and decrypting
  • Potentially something could work mid-February
• Pectra Devnet 5 changes were almost all merged into the main branch

Geth​

• Waiting for the person who had a stack trace to try to reproduce it
• New release process hasn’t been tested yet because there hasn’t been a new upstream release
• There’s a potential issue with the bloom filter code
  • Caused a missing block, but should be an edge case
  • Could also happen in mainline

Reth​

• Crossed Shanghai and everything works fine!
• Can now connect to the p2p network
  • Currently fetching / executing ~2.8m blocks in debug mode directly from peers instead of manually
  • Outgoing connections work fine but incoming are getting blocked
• Syncing until ~600 blocks before Cancun now, will then sync manually during the hard fork
• Not tested with CL yet

CL​

Pectra​

• Are we increasing the blob count on Gnosis Chain?
  • No, keeping at 1/2
• Are we adapting pricing as on Ethereum?
  • Yes, do the same as Ethereum

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Jan 08, 2025

Infrastructure

Gateway​

• About to finish CLs for Ethereum mainnet
• Solved the issue with the 500 requests / batch
  • Should not happen again

Innovation

Shutter​

• Still some issues with the p2p communication between validators and keypers
  • Not optimally connected
  • Looked into by Shutter and Nethermind

Testing

Hive​

• No major update

Ethereum/tests​

• Generating some Shanghai tests with Deb

Client updates

EL​

Nethermind​

• Got 2 reports about missing receipts on archive nodes
  • Potentially found the issue
  • A recovery method could be possible
  • Quite a big edge case

Erigon​

• Shutter
  • Working on libp2p to get the decryption keys

Geth​

• Worked on updating the documentation
• Reported a stack trace in production
  • Seems to happen from time to time during block building
  • In the AuRa code for some reason
• What’s the default gas limit on Geth right now?
  • 30m, should maybe be changed down to 17m
    • For both mainnet and Chiado
• Will likely go to 36m in the next release Releases should work and build Docker image
  • Will test with the new release, or can be tried with an rc

Reth​

• Fixed 2 issues
  • Related to the native bridge contract
• Synced around 2.5m blocks and reached Shanghai
  • Will manually go through Shanghai because some issue happened in Geth during that fork, so might as well play it safe
    • The issue on Geth was linked to the smart contract bytecode size, around 5 hours after Shanghai activation
    • Linked to EIP-170

CL​

Pectra​

• Waiting for EF to release Devnet 5
• Dmytro is testing Pectra for Gnosis Chain in the meantime
  • The deposit scripts and UI don’t work with the devnet
  • Tools are not ready to work with consolidation
    • You can’t deposit more than 1 GNO

Research​

• EIP-7843
  • The SLOT precompile could be used by Shutter to cancel a transaction if it’s not executed in the correct slot
  • This can already be done by using the block timestamp, but that would break if the slot length changes
• EIP-7793
  • The TXINDEX precompile can be used to get the index of the currently running transaction in the block
  • Could cancel a transaction if it’s not executed at the top of the block
    • Or introduce slashing potentially
• Guillaume says that EL devs generally don’t like this type of EIP because this can’t be patched out once people start deploying contracts with them. Introspection makes a lot of things more complicated in the EVM.

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Dec 18, 2024

Infrastructure

Gateway​

• Private access to CL RPCs is in progress
• Public access to Ethereum RPCs will be deployed this week

Innovation

Shutter​

• Dappnode has a package to run a Shutter keyper on Dappnode

  • Might not be published yet, but should be shortly

• Working on improving metrics

• Dappnode also already supports the latest Nethermind version with Shutter

Testing

Hive​

• Mostly focused on consensus spec tests

Ethereum/tests​

• Worked on fixing the test cases
• Everything works with retesteth
• The output of the test cases is going to be tested by Debjit
  • Basically testing Nethermind test cases with Reth

Client updates

EL​

Nethermind​

• Released 1.30.1
  • Small patch release
  • There was an issue where Nethermind didn’t start when set up with systemd
  • There might still be an issue with database corruption

Erigon​

• Nothing specific

Geth​

• The two full syncs on the latest branch have successfully concluded Working with the DevOps team to build releases

Reth​

• Still syncing blocks about the merge
• Manually creating RLPs of 50k - 100k blocks
  • If anything breaks, rebuilding smaller chunks to debug
• Peering doesn’t work because the block hash at genesis is different (pre-merge / AuRa isn’t implemented)
  • The genesis hash could still be hardcoded
  • Peers reject connection because of wrong genesis hash and fork id

CL​

Pectra​

• Devnet 5 should be ready last week of this year for Ethereum according to ethPandaOps
  • We’re targeting first week of January for Gnosis Chain
• Things might move fairly quickly on Ethereum testnets for Pectra

Research​

• EIP-4444
  • How does importing ERA1 files actually work? Is it automatic?
  • Import and export are manual processes Through CLI or RPC
  • The import can be triggered whenever you want for full sync
    • For archive you start by importing every ERA1 file and then sync
  • Sebastian: what are the trust assumptions?
    • The import will work exactly the same way as when you import from p2p, so everything is re-executed and verified
    • You could sync a wrong chain, but at some point you can check with the beacon chain roots or pivot blocks hardcoded in Nethermind that the full chain is correct. At that point you would have to resync again.
  • ERA1 is a first step with Portal potentially coming next
  • The Portal Network will be able to serve arbitrary blocks without having to reimport the full history
  • The Portal Network will also work with light clients
    • Can access headers, receipts and potentially even the state
• ERA1 files are just defined for pre-merge but not for post-merge
• Lion: how would file discovery work
  • On Erigon the torrent hashes are saved on a GitHub repo, so it’s centralized Might be an issue with firewalls for example (e.g. China)
  • There might be a fix for this
• Importing and exporting works with ERA1 files for Gnosis on Nethermind currently
  • The full flow works, i.e. snap sync + full sync from ERA1
  • Next steps: finalize testing and review PR
  • ETA Q1
  • Geth doesn’t have ERA1 import yet or pre-merge history drop, so if Nethermind drops the history then Geth would never be able to sync
  • Even with snap sync
• We need to confirm support from every client
  • Erigon doesn’t rely on this (they use their own snapshots)
  • They’re still going to serve all the blocks by default
  • Nethermind will support it
  • Geth doesn’t yet
  • Reth doesn’t matter yet I’d say, it’s not sufficiently advanced yet
• CL ERA files (from Nimbus)
• https://gbc-era.gnosiscoredevs.io/

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members of the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich and multifaceted and aim to foster innovation within the community.

Watch the record here

Dec 11, 2024

Infrastructure

Gateway​

• No updates

Innovation

Shutter​

• Some issues with the libp2p bootnodes
• Will soon add more keypers
• Plan to make the internal observability Grafana dashboard publicly accessible at some point
  • Still need to hide IPs
• Dashboard
  • https://explorer.shutter.network

Testing

Hive​

• Fixed all failing tests
• Added Reth support
  • Most tests are still failing
  • Allow Deb and others to test their implementation with Hive

Ethereum/tests​

• Depends on Retesteth, which is currently being fixed

Retesteth​

• Fixed some scripts because clients were sending data with extraneous fields
• The Dockerfile had some stable / conflicting dependencies
• Will work on fixing ethereum/tests error messages to be able to fill tests

Client updates

EL​

Nethermind​

• Released 1.30.0
• Almost done with 44s
  • Still fixing issues about exporting and importing ERA files
  • ERA files will be hosted somewhere public (Swarm, IPFS, S3)

Erigon​

• Released erigon3-alpha6
  • Now by default a full node instead of archive node
  • Focused on performance for validating
    • https://erigon.tech/announcing-erigon-v3-alpha-6-focus-on-staking-and-full-node-performance/
• Beta 1 should be released in a few weeks
  • Stability is improving

Geth​

• The full sync on the latest rebase is working fine for now
• Some tester had a database corruption issue on Geth
• Geth is ready to be publicly released

Reth​

• Had issues with post-block processing, the state wasn’t getting committed to the database
  • This was fixed and ~1000 blocks were synced successfully after the merge
• Working on a Reth and Hive integration

Pectra

• All the scripts are ready to deploy a version of Devnet 5 for GC

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Nov 27, 2024

Infrastructure

Gateway​

• Everything’s working well
• Progressing with a Nimbus CL endpoint

Innovation

Shutter​

• Still working on the peer discovery issue

Testing

Hive​

• No updates

Ethereum/tests​

• Working on solving the issues with Nethermind t8n

Client updates

EL​

Nethermind​

• Fixed the most blocking issue for the next release
• Still working on an issue that happens after restarts
  • Difficult to reproduce, fairly rare
  • Seems to happen on very fast machines
• Next release
  • Performance improvements
  • Bug fixes
  • Pectras PRs
  • Shutter in the main branch

Erigon​

• Supports Gnosis Pectra Devnet 1
  • Fix for importing genesis
• Pectra Devnet 4 is live on 2.61.0-beta1
• A v3 version should also work on Gnosis Devnet 1

Geth​

• Finished a full sync on Geth
• Rebased on the latest official release of Geth: 1.14.12
  • Working on a clean PR
  • Able to follow the chain and produce blocks on Chiado
  • Will do a full resync on this latest rebase
• Implemented tracing for system calls
• Cleaned code and removed temporary stuff
• Not sure what Pectra needs to be done on Gnosis
  • https://github.com/gnosischain/specs/blob/master/network-upgrades/pectra.md
• Helped Debjit for Reth
• Missing for a public release
  • The rebase on the right branch
  • Validate everything on the latest rebase with a full sync

Reth​

• Got the post-merge state dump from Geth
• Was able to sync ~70 blocks after importing the state
  • Debugging issues with block execution now, invalid block roots

Research

• Native rollups

  • A debate between Martin and Justin Drake:

• Misc

  • https://eips.ethereum.org/EIPS/eip-7639
    • Generally the consensus is to drop pre-merge history in 6 months
      • Because in 6 months, 2 TB will no longer be sufficient to run a full node
      • Will require support for ERA files, but keep the Portal network optional
        • Nethermind has a PR for ERA files, should be merged soon
          • https://github.com/NethermindEth/nethermind/pull/6547
          • Compatible with both Ethereum and Gnosis
  • We could do the same for Gnosis Chain as a first step
  • Might be problematic for clients like Reth that need to sync the full chain
  • After this, to sync an archive node you’ll need to download and import ERA files manually and the rest would work like it does currently

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Nov 20, 2024

Infrastructure

Gateway​

• Worked on a public RPC for the consensus layer
• Previous version of container deployed by mistake
• Resulted in max 100 batch requests instead of 500
  • Should not happen again

Innovation

Shutter​

• Most of the team was at DevCon
• Working on stability improvements between validators and keepers
• Looking into discoverability algorithms

Testing

Hive​

• Adapted consensus spec tests to Gnosis

Ethereum/tests​

• Discovered the issue on t8ntool but isn’t fixed yet
• Dan and Spencer from the EF joined
  • Want to use EELS/EEST for Gnosis
  • https://notes.ethereum.org/@danceratopz/eest-gnosis
  • This would replace ethereum/tests
  • We would only focus on post-merge tests, as AuRa is too different

Client updates

EL​

Nethermind​

• Working on the next release (delayed because of DevCon)
  • Found critical issues
• Discussing internally about how to do Shutter testing

Erigon​

• No particular updates
• Shutter implementation is going to take 1-2 months

Geth​

• Finished sync up to merge
• 7m blocks to go
• Working on a dump for Debjit on the merge block

Reth​

• Raised some PRs, waiting for review

CL​

Pectra​

Research

• Feedback about Martin’s native rollups
  • Marek thinks that it’s very interesting and would be interested in joining this working group
  • Lukasz has reservations about this idea
    • Mostly social, not technical
    • Ethereum core devs have limited throughput, so decentralizing it through L2s would allow for a faster timeline

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Oct 23, 2024

Infrastructure

Gateway​

• No issues the past week
• Implemented rate limits for batch requests
• In the process of releasing CL endpoints
  • ETA this or next week

Innovation

Shutter​

• Nethermind
  • Engine room release
  • Working on refinements and improvements of the Shutter explorer
  • Testing the latest version on Chiado
    • The stability is looking very good
      • 24 failures out of 8k+
  • Planning for the mainnet release
  • Implementation started by Milen from Erigon
    • Frederik will set a call up

Testing

Hive​

• The execution API suite will get moved
• Some suites rely on t8n

Ethereum/tests​

• Integrating t8n by Nethermind

Client updates

EL​

Nethermind​

• 1.29.1 solves the block creation problem
  • Official release and pushed to Dappnodes etc
  • No missing slots anymore
• Worked on a new Shutter release for 1.29.1, which was released as 1.29.2
• Release 1.30 planned in 1-2 weeks
• Worked with Twinstake
• Should be able to focus more on 44s
• Implemented an endpoint that should allow Nethermind to be used as a block builder

Erigon​

• No particular updates
• Working on devnet 4 and devnet 5
• Will start working on joining Gnosis Devnet 1 as soon as it’s published

Geth​

• This the rebase to the latest 1.14 version (1.14.11)
  • Which should support live tracer
  • Should be ready for Gnosis Devnet 1
    • Missing blob fee collector change
• Snap sync works
• Block building works on mainnet (Gnosis is running 52 validators)
• Still in the process of doing a full sync, but will last at least 2-3 weeks
  • Still pre-merge currently, everything works until there at least

Reth​

• Working on importing the post-merge state from geth or any other client
  • Geth seems to have a command to export the state
    • Guillaume thinks that it might not work
• Waiting for t8ntool

CL​

Pectra​

• Gnosis Devnet 0
  • Based on devnet 3
  • Should probably move to devnet 4 (for Gnosis Devnet 1)
    • Both Nethermind and Erigon agrees, especially because 3 is not compatible with 4 at all

Research

• No updates

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Oct 09, 2024

Infrastructure

Gateway​

• RPC working fine with latest EL versions *Implementing batch rate limits
• Will start working on consensus layer proxy

Innovation

Shutter​

• Nethermind
  • Main Shutter PR has been merged
  • Still working on a few p2p improvements
  • Floodsub has been implemented
  • Peer exchange is being implemented as well
  • Some keys are still not received in time
• Erigon
  • Milen is going to start working on Shutter support

Testing

Hive​

• Focused on CI improvements

Ethereum/tests​

• No updates

Client updates

EL​

Nethermind​

• Testing new release
• Pushing for Pectra
• Talks about gas limit and EIP-4444
• Work on era files for Gnosis will be implemented after 7702

Erigon​

• No updates
• Erigon 2 will support the first fork of Pectra in the end
• Erigon 3 is not yet rock solid stable and the increased RAM requirements might be unfair, especially on Gnosis Chain

Geth​

• Found a bug in the rebase, now syncing the chain
• Will then test block production
• Sync should complete in a few hours

Reth​

• State root mismatch issues after genesis
• Starting on a chain with Shapella but without Dencun
• Only a reward contract call is being made
• Try with 0x0 and system accounts have > 0 balance

CL​

Pectra​

• Potentially looking at Q1 2025

Research

• No updates

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Oct 02, 2024

Client Team Updates

EL​

• Nethermind:

  • Testing a new release:
    • performance improvements, fixes, engine_getBlobs ->
    • might be useful for gnosis validators a little bit who are struggling with bandwidth,
  • A lot of work on Pectra
    • Generally a lot of progress in the performance area

• Erigon:

  • Busy with Erigon 3
    • Small forks for Pectra
    • Forcing to users from Erigon 2 to Erigon 3
    • User have to resync
    • Sync time will significantly reduce
  • General availability Erigon 3
    • Changing UX
    • Config and storing everything will change
    • Trade-off: optimizing RAM
    • Dealing with hive issues
      • will fix it soon

• Geth:

  • Still debugging the rebase

• Reth:

  • Welcome Debjit!
  • Fixed the differences between production and verification

Pectra

• Which path should we follow for Gnosis
  • See Ethereum ACD
  • https://github.com/gnosischain/specs/pull/51
• When should we start testing?
  • Gnosis specific devnet
  • Nethermind: we are starting slowly

Chain Infra

• Gateway
  • Gateway: Created new extended feature batch limit.
    • allowing 500 requests
  • RPC works well with the latest version.

Innovation

• Shutter
  • Finalized engine room. Scheduled next week.
  • Explorer.shutter.network is ready
  • Working on next iteration. Tomorrow will be live
    • New deployment on Chiado
  • Nethermind:
    • peer discovery testing is underway
  • Erigon:
    • Team working on for implementation
    • New team member!

Testing

• Hive
  • Updated new test
• Ethereum/tests:
  • Fix some operational issues
  • Still working on it

Welcome to the Gnosis Core Devs weekly gathering. Every Wednesday, key members from the Gnosis team, contributors, and various team representatives convene to discuss, collaborate, and update one another on the Gnosis ecosystem's progression.

Participants represent teams:

Erigon, Gateway, Nethermind, Geth, Gnosis DevOps, Gnosis Core Devs, Gnosis Comms team.

With a diverse set of voices present, our discussions are rich, multifaceted, and aim to foster innovation within the community.

Watch the record here

Aug 28, 2024

Client Team Updates

EL​

• Nethermind:

  • Working on the next release

    • Improvement on performance

    • Probably mid next month

• Erigon:

  • Nothing new

• Geth:

  • Started rebasing from 1.13 to 1.14 (which actually changed many things in the miner)

    • 1.14 can no longer sync pre-merge chains

    • So one has to sync with 1.13 and then update to 1.14

• Reth:

  • No update

Chain Infra

• Gateway

  • No major updates

  • Everything is working well except for one quick restart

Innovation

• Shutter

  • Still working on the last 10% in missed transactions

    • New keyper version with floodsub discovery mechanism

  • Also working on the Shutter explorer

Testing

• Hive

  • Good progress

  • Fixed all Nethermind tests

  • 2 new tests suits (13 new tests) for Nethermind

  • Fixing in progress for Erigon

  • New PR with new engine suite

• Ethereum/tests

  • Started to integrate the Nethermind implementation

  • Might require a new repo