Planet

Last updated: 2026-10-05 18:45:01 UTC

Valuable News – 2026/10/05

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

New x11/bosd FreeBSD Tool for OSD.
https://linkedin.com/posts/dteske_freebsd-ugcPost-7510131549820932096-CPKb/

Gaffer 1.7.3.0 Now Runs Natively on FreeBSD.
https://linkedin.com/posts/martin-f-473405380_freebsd-gafferhq-imageengine-ugcPost-7509773139636092928-0RtV/

HardenedBSD 08-09/2026 Status Report.
https://hardenedbsd.org/article/shawn-webb/2026-09-27/hardenedbsd-august-september-2026-status-report

FreeBSD Git Weekly: 2026-09-21 to 2026-09-27.
https://freebsd-git-weekly.tarsnap.net/2026-09-21.html

FreeBSD Kernel Frame Unwinding with DWARF CFI.
https://minsoo.io/p/freebsd-kernel-frame-unwinding-with

FreeBSD Foundationals: rc.d – Service Framework That Fits in Your Head.
https://blog.hofstede.it/freebsd-foundationals-rcd-the-service-framework-that-fits-in-your-head/

FreeBSD Running on Apple M6 (T8152).
https://linkedin.com/pulse/freebsd-apple-m6-t8152-bring-up-update-alan-kong-d4w5c/

The mergedotpkg is FreeBSD pkg(8) Companion for Merging .pkgnew/.pkgsave Files.
https://github.com/0mp/mergedotpkg

Install Ubuntu 26.04 Server in OpenBSD vmm(4) VM.
https://rsadowski.de/posts/2026/vmm-ubuntu26-setup/

FreeBSD pkg(8) Plugin to Create ZFS Boot Environment Before Each Package Install/Upgrade/Remove.
https://github.com/usenix17/pkg-be-plugin

Mocka Desktop from GhostBSD Aims to Rewrite and Replace Each MATE Component.
https://mocka-desktop.org/

FreeBSD Journal 2026 Q3.
https://freebsdfoundation.org/our-work/journal/browser-based-edition/production-deployments/

GhostBSD Replaces MATE with New Mocka Desktop.
https://itsfoss.com/news/ghostbsd-new-desktop/

FreeBSD Foundation 2026/07 Laptop Update.
https://freebsdfoundation.github.io/proj-laptop/monthly-updates/2026-07

FreeBSD Release Cycles and Enterprise Requirements: Why LTS Matters.
https://klarasystems.com/articles/freebsd-release-cycles-enterprise-requirements-lts/

Reduce Duration of Solaris SRU Updates Using JomaSoft VDCF.
https://jomasoftmarcel.blogspot.com/2026/09/reduce-duration-of-solaris-sru-updates.html

New Features in Solaris 11.4 SRU95.
https://jomasoftmarcel.blogspot.com/2026/09/new-features-in-solaris-114-sru95.html

Apache Module Integrating blocklistd(8) on FreeBSD.
https://github.com/koron/mod_blocklist

What About ThinkCentre M710q/M720q and *BSD.
https://tumfatig.net/2026/what-about-thinkcentre-m710q-m720q-and-bsd/

Every Read on QNX is Message.
https://vivianvoss.net/blog/qnx-message-passing

What is New in httpd(8)/relayd(8) on OpenBSD.
https://rsadowski.de/posts/2026/update-relayd-and-httpd/

Running Elixir Graphics and ML Tools on FreeBSD.
https://people.freebsd.org/~dch/posts/2026-05-25-elixir-graphics-ml/

SmartOS: Illumos Way of Thinking About Servers.
https://it-notes.dragas.net/2026/09/28/smartos-the-illumos-way-of-thinking-about-servers/

GSoC 2026 Reports: Improving and Stabilizing Racoon 2 IKE Daemon in NetBSD.
https://blog.netbsd.org/tnf/entry/gsoc2026_racoon2

Self Hosted HTTP Tunnels with SSH and Nginx.
https://vincent.bernat.ch/en/blog/2026-http-over-ssh

FreeBSD with AMD Radeon RX 9060 XT.
https://reddit.com/r/freebsd/comments/1wujkv8/freebsd_9060_xt/

Cron for Laptops vdcron 0.9 Now Survives Shutdowns/Power Loss/Reboots.
https://vincentdelft.be/post/post_20261003

OpenBSD Devs Reject uutils Coreutils Port over Licensing and Compatibility Concerns.
https://news.lavx.hu/article/openbsd-developers-reject-uutils-coreutils-port-over-licensing-and-compatibility-concerns

UNIX/Audio/Video

Worry Free Upgrades with ZFS Boot Environments on FreeBSD.
https://youtube.com/watch?v=MfHCXEzRfao

I Switched from Linux to FreeBSD. Here is How That Went.
https://youtube.com/watch?v=Rk5kJ2iWYaU

Trying to Play Games on FreeBSD. [Polish]
https://youtube.com/watch?v=kXHafxT9UOI

OpenBSD Says No to Rust Clones.
https://youtube.com/watch?v=p37cN6Me1zg

BSD Now 683: GeFS.
https://www.bsdnow.tv/683

Life

It Took Me 33+ Years to Realize what George Carlin Will Tell You in 10 Minutes.
https://youtube.com/watch?v=PpN80Ab6Jlo

Apple Removes BitChat from India App Store at Government Request.
https://reclaimthenet.org/apple-removes-bitchat-india-government-request

Other

IFPI Wants Open Source YouTube Downloader yt-dlp(1) on EU Piracy Watch List.
https://torrentfreak.com/ifpi-wants-open-source-youtube-downloader-yt-dlp-on-eu-piracy-watch-list/

Era of Software Quality or Era of Ostriches?
https://blogs.gnome.org/mcatanzaro/2026/10/02/the-era-of-software-quality-or-the-era-of-ostriches/

Unofficial Fan Made Knights and Merchants Mod Release Candidate.
https://kamremake.com/2026/09/announcing-release-candidate/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

October 2026 Update: Getting Ready for the EU Cyber Resilience Act

Post by FreeBSD Foundation via FreeBSD Foundation »

Please note that the information provided in this document is for informational purposes only and does not constitute legal advice.

Earlier this year, the Foundation launched its Cyber Resilience Act (CRA) Readiness project. The launch blog post outlines the reasons for the project and what it seeks to achieve. 

A lot of progress has been made in the project since February which we would like to share with you. First, a reminder that the project is being documented on a GitHub repository, with monthly updates and meeting notes available for you to read. 

What is the CRA? – A quick refresher

Note: this is a very brief overview. There is a lot of detail in the regulation which is not given here.

  • The EU Cyber Resilience Act is a new EU regulation. 
  • It affects anyone selling Products with Digital Elements (PDEs) in the EU. 
  • It mandates that PDEs are managed through a secure software lifecycle and that actively exploited vulnerabilities must be reported to the European authorities.
  • Its goal is to improve the security of software (and hardware with software installed) for users residing within the EU, whether individuals or organizations. 
  • The regulation is enforced through various mechanisms, including non-compliance fines for manufacturers of up to €15M or 2.5% of global revenue. 
  • The regulation is already in place, with a phased rollout. Manufacturers are already obliged to report actively exploited vulnerabilities within 24h of discovery, and after Dec 11, 2027 all PDEs on the market must meet “Secure by Design” obligations and have an CE mark affixed.

How does the CRA affect FreeBSD? 

FreeBSD vendors serving the EU market have a responsibility to ensure that the implementation of FreeBSD they include in their products meets the CRA’s requirements for “Secure by Design” and “Secure by Default”. They are also responsible for reporting any actively exploited vulnerabilities in their entire codebase which naturally includes FreeBSD code.

The FreeBSD Project itself, along with its contributors, is not directly regulated by the CRA but may expect to see a change in focus for manufacturers who are seeking to meet their CRA obligations. There is a little over a year remaining for manufacturers to get their SBOMs in order, and make any other changes needed so that their PDEs have a secure software lifecycle. Their reporting obligations are already in force, so a critical incident (with the chance of large fines) could occur at any time. 

As the upstream project, FreeBSD may start to experience some behavioral changes in downstream manufacturers, for example:

  • Pressure to engage with manufacturers who are up against tight vulnerability report and remediation deadlines, or who are looking for patches in a hurry.
  • Manufacturers looking for information on FreeBSD secure development practices, or who are looking for a FreeBSD SBOM.
  • Manufacturers looking for ways to reduce risk in FreeBSD or its dependencies or who are looking for alternatives to FreeBSD.
  • Manufacturers coming with questions about how FreeBSD works or is developed, or who want changes to how FreeBSD works or is developed.

What is the Foundation’s role in this?

The FreeBSD Foundation exists to support the FreeBSD Project. Its longstanding and extensive role in supporting the Project means it is classified as an “open source software steward” under the CRA. There are a few lightweight obligations under the CRA for stewards, but there are no fines. Where stewards are involved in development work or managing infrastructure, they have some reporting responsibilities but these do not come into force until Dec 11, 2027. They are also required to provide a cybersecurity policy to Market Surveillance Authorities (MSAs) on demand. 

Getting ready to meet these responsibilities has been part of the Foundation’s Cyber Resilience Act (CRA) Readiness project, but the project has been scoped to go much further and ensure that the FreeBSD Project and its community are ready to support and engage with manufacturers as they seek to meet their CRA obligations.

Project progress to date

Reminder: the project is being documented on a GitHub repository, with monthly updates and meeting notes available for you to read. 

Foundation readiness

Manufacturer outreach program

The Foundation has been contacting manufacturers to understand what they are doing to prepare for the CRA and to seek their support for FreeBSD. The engagement levels have been mixed, with European vendors being most advanced in their understanding and interest.

Steward obligations

The Foundation has created its Cybersecurity Policy, an internal document that captures existing relationships and processes in the FreeBSD Project and can be provided to the MSA as neededWe have also created an internal live cybersecurity incident protocol for Foundation staff to follow in case of an incident which needs to be reported under the CRA.

The Foundation has been preparing to use the European Union Agency for Cybersecurity (ENISA) reporting platform. This is the platform that manufacturers are already using to report actively exploited vulnerabilities, and it will be opened up to stewards on Dec 11, 2027.

Open Source solidarity

We have been engaging with the Eclipse Foundation’s Open Regulatory Compliance (ORC) working group to keep up with the latest news from the European Commission (EC), share perspectives, and support other projects.

FreeBSD Project readiness

Security Team Resilience

The Foundation has invested in increasing the resilience, capacity, and CRA expertise of the Security Team by sponsoring Pierre Pronchery to join as a new member. His dual role on the team and as a Security Engineer for the Foundation has been pivotal in facilitating the flow of information in both directions. He is currently working on a CRA live incident information pack for the Security Team to reference in case of a live incident.

Core Team engagement

The Foundation has been keeping the new Core Team (which was formed in June) up to date with the work in flight, and sharing CRA information so that the Core Team can begin to include CRA considerations in their work.

SBOM tooling for FreeBSD

A large part of the readiness project has been dedicated to developing the tooling for creating an SBOM at build time for FreeBSD. SBOMs (Software Bills of Materials) are required by manufacturers under the CRA and form the backbone of license compliance and vulnerability management. This has proven to be a complex and challenging project which has many unknown unknowns. The progress has not been linear but the solution is becoming very robust and comprehensive, and will prove to be an excellent asset to FreeBSD over the coming years. Work is ongoing and will be available in FreeBSD 16, and also in point releases of FreeBSD 14 and 15 when ready.

FreeBSD community readiness

Information sharing

The Foundation has been working hard to understand the CRA and share FreeBSD-specific information about it. Key outputs include:

Legislative engagement

We have been sharing known open consultations on legislation with the community via mailing lists, and on a GitHub reference page. We have had some really supportive community members who have collaborated on providing feedback to the EC on their draft public guidance.

What’s next and how can you help?

The Foundation’s project has achieved the majority of its immediate goals, but the CRA is here to stay and FreeBSD will feel its influence over the coming months and years. The Foundation remains committed to supporting FreeBSD through this phase of its development.

Our main focus going forward will be to work with the Core Team, Security Team, and manufacturers to make improvements to FreeBSD’s security posture that will in turn make CRA compliance easier. Keep an eye on the monthly updates to track progress.

The number one thing you can do to help is to learn about the CRA – so thank you for reading this post. If you have any questions feel free to ask, by emailing cra@freebsdfoundation.org. Your questions help us to improve our work.

If you are a manufacturer affected by the CRA, we would love to hear from you and understand how you are preparing and what you need from FreeBSD. Where concrete changes are needed, we would be happy to coordinate a joint effort supported by like-minded manufacturers.

Donations in support of the CRA readiness work are always appreciated and will help us to continue supporting this important work

The post October 2026 Update: Getting Ready for the EU Cyber Resilience Act first appeared on FreeBSD Foundation.

Top

Expat 2.9.0 released, fixes two vulnerabilities

Post by Sebastian Pipping via Hartwork Blog »

For readers new to Expat:

libexpat is a fast streaming XML parser. Alongside libxml2, Expat is one of the most widely used software libre XML parsers written in C, specifically C99. It is cross-platform and licensed under the MIT license.

Expat 2.9.0 was released a few hours ago. The key motivation for cutting a release now was to get the security fixes and also new features into the hands of Expat's users.

The two security issues are:

  • CVE-2026-77214 — a potential out-of-bounds read with function XML_ParseBuffer due to insufficient validation of the length parameter,
  • CVE-2026-102633 — an integer overflow in memory management code that affects 32-bit platforms.

If the link for CVE-2026-77214 still results in 404 NOT FOUND by the time of reading, that's because publishing is still in progress.

For CVE-2026-102633 I would like to thank Filippo Tedeschi and Matthew Fernandez. For CVE-2026-77214 I would like to thank Filippo Tedeschi once more, as well as Fabian Wahle (Hap Security).

One of the two new features is the so-called new "Properties API". You can think of properties as the control knobs of a parser. There are five properties at the moment in Expat 2.9.0…

  • Two for the parameters of the allocation tracker/limiter, which prevents maliciously excessive use of memory:
    • XML_PROP_ALLOC_TRACKER_ACTIVATION_THRESHOLD of type uint64_t
    • XML_PROP_ALLOC_TRACKER_MAXIMUM_AMPLIFICATION of type double
  • Another two for the parameters of the protection against billion laughs attacks:
    • XML_PROP_BILLION_LAUGHS_ACTIVATION_THRESHOLD of type uint64_t
    • XML_PROP_BILLION_LAUGHS_MAXIMUM_AMPLIFICATION of type double
  • One for the state of reparse deferral:
    • XML_PROP_REPARSE_DEFERRAL_ENABLED of type XML_Bool

…and there are getters and setters to adjust or retrieve their values:

The new API was introduced to:

  • allow Expat to add more properties — both security-related and non-security ones — in the future (without need to also add new related getter and setter functions for each of them as was the case in the past),
  • allow users to read (not just to set) the current values, and
  • allow users to distinuish the different sources of error beyond a mere "did not work".

The other feature being introduced with release 2.9.0 is the 64-bit location API. The old location API, that is made up of the five functions…

…has the core problem that it uses datatypes int, XML_Index and XML_Size. These data types are not big enough to work with XML content of more than 4 GiB in size: at some point they would wrap around. Macro XML_LARGE_SIZE that increased the size of these datatypes was introduced with Expat 2.0.0 in 2006 to address that problem, but the macro breaks ABI compatibility with anyone that is expecting the smaller datatypes, splitting the world in two, and is not commonly activated — the sitation was not great.

To improve on that situation, Expat 2.9.0 now deprecates macro XML_LARGE_SIZE and offers five new 64-bit versions in return, note the "64" at the end:

This approach now allows everyone that previously relied on the XML_LARGE_SIZE macro to return to mainstream Expat while having access to 64-bit location information that supports XML content beyond 4 GiB in size without wrapping around.

As usual, there are non-security non-feature improvements included with this release. For more details about this release, please check out the change log.

It should be noted that it was the Open Source Sabbatical of the City of Munich that allowed me to focus on libexpat for this release through its funding — thank you! We're in the third month of hopefully six months total; there continues to be plenty to do. Wish me luck!

If your business relies on Expat beyond January 2027, please consider funding the maintenance of Expat to ensure its health and security for you and others. Thank you!

Thanks to everyone who contributed to this release of Expat!

If you maintain Expat packaging, a bundled copy of Expat, or a pinned version of Expat, please update to version 2.9.0. Thank you!

Sebastian Pipping

Top

Dutch Police Arrest ‘Reformed’ Hacker in Shiny Hunters Investigation

Post by Brian Krebs via Krebs on Security »

Authorities in the Netherlands have arrested a 24-year-old convicted cybercriminal on suspicion of aiding in data thefts and extortions by the prolific hacker group ShinyHunters. In the days immediately following the suspect’s arrest, remaining ShinyHunters members dramatically escalated their attacks, stealing highly sensitive data from the FBI and extorting the Russian ransomware group Cl0p.

According to three sources familiar with the matter, the Dutch man arrested by authorities this month is Pepijn van der Stap, a convicted cybercriminal from Almere and Lelystad in the Netherlands. Van der Stap was previously convicted in 2023 in connection with a string of data thefts and extortions that prosecutors said earned between €1.5 million and €2.7 million.

At his trial in late 2023, van der Stap admitted that he lived a Dr. Jekyll and Mr. Hyde existence, secretly using the hacker handle “Umbreon” to extort victims and post their data on English language hacking communities like the now-defunct RaidForums and Breached. By day, however, van der Stap was working as a software engineer at the Amsterdam-based cybersecurity startup Hadrian, while volunteering at the Dutch Institute for Vulnerability Disclosure (DIVD), a nonprofit security research group.

Pepijn van der Stap’s alter ego “Umbreon” selling a database on RaidForums, offering information on 2.3 million people from The Netherlands in September 2021. This user’s avatar is a depiction of the Pokemon character Umbreon. Image: KELA.

Van der Stap confessed to his data theft and extortion activity, and was sentenced to four years in prison (one of which was suspended). During his trial, van der Stap opted to remain in custody for a time rather than at home, saying he could not find better treatment on the outside for his ongoing psychological issues, which he claimed included PTSD related to childhood trauma. He was released from prison in December 2025.

In an interview with KrebsOnSecurity on September 9, 2026, Van der Stap cast himself as a reformed hacker who was trying to turn his life around and make a positive contribution to society. Van der Stap is currently employed as offensive security lead at the Dutch company Neo Security, which did not respond to requests for comment.

Van der Stap said he was still dealing with civil lawsuits and restitution related to his previous cybercrime victims, and that he was trying his best to make amends. But not long after that interview, the Dutch hacker abruptly stopped replying to messages. Efforts by others close to him also repeatedly failed to elicit a response for the past two weeks.

The LinkedIn profile for Pepijn van der Stap.

According to two sources with knowledge of the matter, Van der Stap was arrested by Dutch authorities on or around September 16, and has been held in custody for questioning since. One source said a colleague of theirs personally witnessed Dutch authorities carting items out of Van der Stap’s residence.

Authorities in the Netherlands have been asking the public for help in identifying the voice in a recorded telephone call from February 2026 in which a native Dutch-speaking ShinyHunters member social engineered their way into Odido, the nation’s largest mobile telecommunications provider. In that intrusion, ShinyHunters tricked an Odido employee into logging in at a spoofed website, and then used that access to steal data on more than 6.2 million Dutch people.

Responding to Dutch news media, ShinyHunters confirmed that the suspect in the audio clip is indeed a member of the hacker collective.

“Our team member has our full support – emotionally, mentally, and financially,” the hackers said. “Everything has been arranged, including a criminal defense lawyer. We do not look down on our staff and members; we take excellent care of them,” reads a statement ShinyHunters shared with NL Times. It remains unclear if the Dutch police have matched the Odido caller to a confirmed real-life identity. The Dutch police unit handling the Odido incident did not respond to requests for comment.

The group also lashed out at the authorities in the Netherlands. “The Dutch police will need all the luck in the world – and everyone’s prayers – if they want to catch him before we carry out another large-scale data theft in the Netherlands,” the ShinyHunters statement said. “Frankly, the Dutch police are a big joke; they are incapable of doing anything. Incompetent. Irrelevant. Unimportant. Useless.”

FBI, CL0P HACKS

Just days after sources say Van der Stap was detained by Dutch authorities, ShinyHunters claimed credit for an unusually brazen breach at the FBI’s job application site apply.fbijobs.gov. According to reporting from 404 Media, the data stolen from the FBI site includes Social Security numbers and personal information on more than 5,000 officials.

404 Media and Reuters reported the FBI data included each person’s job title or team, such as special agent, threat intake examiner, major cybercrimes unit, and those investigating cyber threats from foreign state-backed actors. Reuters examined documents shared by ShinyHunters and found they included sensitive psychiatric and medical files of FBI staff. The FBI issued a brief statement confirming the hack.

ShinyHunters said it gained access to the FBI site and other victims by exploiting a recently patched vulnerability (CVE-2026-35273) in PeopleSoft, a software-as-a-service platform from the software giant Oracle that is broadly used by companies to manage hiring and human resources, benefits and payroll. Oracle quickly issued a fix for the Peoplesoft vulnerability that ShinyHunters reportedly began exploiting as a zero-day in June, and at the time Mandiant released web application firewall rules intended for organizations who couldn’t apply the security update quickly enough.

But on Friday, BleepingComputer reported that ShinyHunters used a URL-encoding trick to bypass Mandiant’s suggested web application firewall rules designed to mitigate the threat from the PeopleSoft flaw. In a report released Sept. 25, security experts at Mandiant and the Google Threat Intelligence Group (GTIG) confirmed that ShinyHunters had mass-exploited the PeopleSoft vulnerability to steal data from dozens of systems across a range of industries, including higher education, technology, healthcare, agriculture, transportation and government.

Van der Stap’s former hacker alias Umbreon was hidden in plain sight throughout the imagery ShinyHunters used to spread news about the FBI hack: The defacement image that ShinyHunters left behind on the hacked FBI jobs site included an ASCII art design featuring the Pokemon character Umbreon. The message at the top read, “This site has been seized by ShinyHunters. rooting your systems since ’19 ;)” The image appears identical to a defacement message ShinyHunters used in their 2020 hack of the English-language cybercrime community Hackforums.

The defacement message left by ShinyHunters on the FBI jobs site included an ASCII art rendition of the Pokemon character Umbreon. Image: Bleeping Computer.

Multiple sources close to the ShinyHunters investigation said the group’s recent risky attacks against the FBI and one of Russia’s most venerated ransomware groups amounted to a major pivot away from the more measured tenor of the hacking gang’s operations. Those sources said the sudden shift came about after ShinyHunters was taken over by a teenage cybercriminal from Amman, Jordan who goes by the nickname Rey and operates as part of a cybercrime group called ScatteredLapsussHunters (SLSH), which experts say is an amalgamation of three hacking groups — Scattered Spider, LAPSUS$ and ShinyHunters.

Those sources said Rey had an ongoing beef with the Dutch hacker over control of the ShinyHunters brand and data, and that the inclusion of the oversized Umbreon Pokemon image in the FBI jobs site defacement was likely an attempt by Rey to pin the hack on the Dutchman.

Rey was first publicly identified by the cybersecurity firm KELA in March 2025. In advance of our November 2025 profile of Rey, KrebsOnSecurity messaged Rey’s father and asked for permission to interview his teenage son. Rey’s dad merely forwarded the message to his son, who admitted to participating in ransomware attacks and said he was trying to extricate himself from the SLSH hacker group.

BLAMING UMBREON

Immediately after news of the FBI jobs site hack was picked up in the media, Rey’s main account on Twitter/X (Ryan Moran/@rmoskovy) was taunting the Cl0p ransomware group and the FBI, crudely depicting them as the twin towers in New York being struck by planes labeled “cl0p drama” and “fbi breach claim.” In the foreground of the city is the giant Pokemon figure of Umbreon.

A taunting meme uploaded to Twitter/X by Rey’s now-defunct account on Sept. 22. A giant float-sized version of the Pokemon character Umbreon can be seen in the bottom left.

On Sept. 24, KrebsOnSecurity again contacted Rey’s dad, asking to interview him and his son for a story on Rey’s apparent ascendency as the head of ShinyHunters. Just hours after that request, Rey deleted his longtime Twitter/X account. Meanwhile, Rey’s dad, who works for the Royal Jordanian Airlines, has failed to respond to a half-dozen emailed requests for comment about his son’s alleged activities.

Where does the bad blood between SLSH and ShinyHunters come from? According to a story in Wired this month, ShinyHunters and SLSH members briefly partnered earlier this year to help better monetize important stolen credentials collected by TeamPCP, an upstart group that was having great success compromising global code supply chains with malicious software but hadn’t been able to profit much from their stolen data (two alleged leaders of TeamPCP were arrested last month in Australia, and in an interview the TeamPCP leader claimed they made just $20,000).

The Wired story noted how Mandiant had infiltrated TeamPCP and was secretly responsible for having the crime group’s stolen credentials burned so quickly: Mandiant was secretly feeding those credentials to the major cloud providers like Amazon and Microsoft, who quickly invalidated the stolen keys. Meanwhile, the formerly cooperating hacker groups began to blame one another for causing the credentials to become worthless.

Wired’s Andy Greenberg reported that a few weeks after partnering with TeamPCP, “ShinyHunters went rogue, carrying out its own extortions with TeamPCP’s credentials but without giving the supply-chain hackers their cut.”

Mandiant researcher Austin Larsen told KrebsOnSecurity earlier this month that ShinyHunters has been enjoying a successful extortion spree so far this year, and is on track to pull in nearly $100 million in extortion payments from cybercrime victims in 2026.

Van der Stap claims he was never motivated by money and that his earlier hacker activity was driven by a desire to have the world’s most complete collection of stolen databases. Speaking with reporters from Bloomberg in 2024, Van der Stap said that singular focus in turn fueled his desire to carry out cyberattacks.

“The hacking was very easy for me, and it wasn’t a compulsion,” he told Bloomberg. “My habit was collecting. Collecting data, organizing data, downloading data, creating folders.”

DIVD, the nonprofit security research group where Van der Stap previously served as a volunteer, disclosed on LinkedIn last week that the organization was dealing with an internal cybersecurity incident that appears to have involved the malicious use of artificial intelligence. DIVD has released few details about that incident, but a spokesperson for the nonprofit told KrebsOnSecurity it does not appear related to ShinyHunters, nor are there any signs the matter involves the work of a previous volunteer.

Update: 3:44 p.m. ET: Corrected Van der Stap’s age, which is 24 (not 23).

Update, 4:54 p.m. ET: The Dutch police have confirmed the arrest of a 24-year-old in connection with the ShinyHunters investigation. In a statement on Twitter/X, the Dutch police said the man will appear on Tuesday, September 29 before the chambers of the Rotterdam District Court, and that it will provide more information tomorrow.

Sept. 29, 9:42 a.m. ET: The Dutch news outlet RTL reports that investigators suspect Van der Stap tried to orchestrate at least two murders. RTL reported the two murders were allegedly to be committed abroad, and that there are indications the suspect gave the order for this.

The FBI released a short video message on the ShinyHunters investigation from Brett Leatherman, assistant director of the FBI’s cyber division, who thanked Dutch law enforcement partners for their assistance and urged remaining ShinyHunters members to turn themselves in.

“Arrests have a way of changing who is willing to talk, and seized infrastructure has a way of showing us who’s left,” Leatherman said. “The longer you stay in this, the more we learn about you. You know how to find us, and we know how to find you. I suggest you reach out to us while the choice is still yours.”

Top

Valuable News – 2026/09/28

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

Testing compat_linux(4) on NetBSD with Linux Test Project.
https://lwn.net/Articles/1094310/

PDP-11/83 Live System Overview with 2.11BSD OS.
https://pdp1173.com/

FreeBSD Git Weekly: 2026-09-14 to 2026-09-20.
https://freebsd-git-weekly.tarsnap.net/2026-09-14.html

GhostBSD Renames Backup Station to BE Station.
https://github.com/ghostbsd/be-station/pull/24

Analog Audio on Intel Broadwell-U Platforms with Realtek ALC3263 Codec on FreeBSD.
https://github.com/spagu/acpi_intel_sst

Intel RAPL Package Power Limits and Power Measurement for FreeBSD.
https://github.com/spagu/intel_rapl

Patch for FreeBSD ig4(4) ACPI Enumerated Intel LPSS I2C Controllers.
https://github.com/spagu/freebsd-ig4-lpss

Setup FreeBSD Testing on GitHub Actions.
https://github.com/sagemath/sage/issues/35129

Brave Now Runs Natively on FreeBSD.
https://reddit.com/r/freebsd/comments/1wm8y3s/brave_natively_on_freebsd/

OnemanBSD Self Containing OS.
https://bialamusic.com/onemanBSD/

vibe Coded FreeBSD Kernel Module That Uses GitHub Downtime as Entropy Source.
https://github.com/DominoTree/ghrandom

Call for Testers: AI Vibe Coded JFS for FUSE.
https://lists.freebsd.org/archives/freebsd-hackers/2026-September/006894.html

Biggest ZFS Misconfigurations and How to Fix Them: Part 1.
https://klarasystems.com/articles/zfs-misconfigurations-how-to-fix-part-1/

New kldinfo(8) FreeBSD Tool to Display Module Objects Deps and Version Info.
https://lists.freebsd.org/archives/freebsd-current/2026-September/010855.html
https://github.com/pointcheck/kldinfo

The zstd(1) Decompresses at One Speed.
https://vivianvoss.net/blog/zstd-one-speed

Launching FreeBSD/EC2 KDE Desktop AMIs.
https://daemonology.net/blog/2026-09-24-Launching-FreeBSD-EC2-desktop-AMIs.html

2026 Became Year of FreeBSD Desktop on Amazon/AWS Cloud.
https://theregister.com/os-platforms/2026/09/25/2026-year-of-freebsd-desktop-aws/5299025

NetBSD 10.2 Announced.
https://netbsd.org/releases/formal-10/NetBSD-10.2.html

Linux Programs Look at Things in /proc More Than You Would Expect.
https://utcc.utoronto.ca/~cks/space/blog/linux/ProcGetsUsedByPrograms

11W FreeBSD NAS.
https://interfacecraft.online/blog/2026/11w-freebsd-nas/

Mocka Dock is Modern Taskbar Dock for MATE and Mocka Desktops.
https://github.com/mocka-desktop/mocka-dock
https://x.com/Ericbsd/status/2103960850759401503

Will FreeBSD 15.2 Finally Ship Native KDE Plasma Installer?
https://pbxscience.com/will-freebsd-15-2-finally-ship-a-native-kde-plasma-installer/

Solutions for Kernel Related pkg(8) Packages on FreeBSD.
https://linkedin.com/feed/update/urn:li:activity:7510081590363058178/

NetBSD: Playing with Disklabels.
https://movq.de/blog/postings/2026-09-25/0/POSTING-en.html

Fix 'Newer FreeBSD Version for Package pkg(8)' in FreeBSD Jail.
https://www.gaelanlloyd.com/blog/freebsd-fix-newer-freebsd-version-pkg-error/

OpenBSD vmm(4)/vmd(8) Gain Support for Multi Processor VMs.
https://undeadly.org/cgi?action=article;sid=20260927115632

OpenBSD Gains New Networking rtrd(8) Daemon.
https://undeadly.org/cgi?action=article;sid=20260927120124

OpenBSD-current Has Moved to 8.0 Version.
https://undeadly.org/cgi?action=article;sid=20260927120854

Extended System Information Script for FreeBSD.
https://forums.freebsd.org/threads/x-fetch-gawk-pipe.103917/

UNIX/Audio/Video

Making of BSD: Actual World First Open Source OS?
https://youtube.com/watch?v=tuI2wX3ol2o

BSD Now 682: NAS NAS NAS.
https://www.bsdnow.tv/682

Hardware

Reality Check for IBM POWER versus X86 Solutions.
https://linkedin.com/feed/update/urn:li:activity:7508650853973741569/

Replacing Battery on CyberPower UT650EG/UT850EG UPS.
https://ounapuu.ee/posts/2026/09/24/replacing-the-battery-on-a-cyberpower-ut650eg-and-ut850eg-ups/

Other

SQLite in Production with WAL Mode.
https://reddit.com/r/Python/comments/1wowf4q/sqlite_wal_mode/

It is Not That I Like COBOL. I Like Software That Leaves Me Alone.
https://tara.sh/posts/2026/2026-09-21_i_dont_like_cobol/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

HardenedBSD August / September 2026 Status Report

Post by HardenedBSD via HardenedBSD »

I'm writing this status report a little early because this coming week (the week of the 28th of September 2026) is HardenedBSD quarterly release engineering week. On that note, I plan to cherry-pick a few "commits that smell like security" that FreeBSD recently made against their main branch into our 15-stable branch. I suspect over the next week or two, we might see a FreeBSD Security Advisory and/or Errata Notice.

In src:

  1. Explicitly dissuade from pkgbase use (in bsdinstall)
  2. Update /etc/os-release and /var/run/os-release
  3. Enable -ftrivial-var-auto-init=zero for video(4)
  4. fix links in usr.bin/login/motd.template
  5. Document rejection of AI/LLM/etc generated works
  6. fix broken references in hardening(4) man page
  7. Harden ssh_config(5) by disabling compression and TCP keepalives by default
  8. Integrate -fbounds-safety in world with MK_BOUNDS_SAFETY, disabled by default
    • The flag is still experimental, so it's actually spelled `-Xclang -fexperimental-bounds-safety`

In Ports:

  1. Fix broken dependency for math/libformfactor

I'd like to chat a little bit about -fbounds-safety. This feature is not used anywhere in the base OS. And, the feature is currently disabled by default in HardenedBSD. However, adding the plumbing will allow folks downstream from us to more easily integrate that feature on their end. Folks building things with HardenedBSD can more easily use this experimental feature in their own code. Right now, we only integrated the flag with userland. Once the clang/llvm folks consider the feature production-ready, I'll duplicate the logic to apply to the
kernel and kernel modules.

I view it similar to Capsicum: it requires direct integration in the project's codebase. These approaches tend to feel heavy-handed to me. However, providing the plumbing in HardenedBSD will enable others to make the decision for themselves on whether (and where) it makes sense to adopt.

Infrastructure:

I applied updates across our infrastructure. I migrated two environments from VMs to jails: rad.hardenedbsd.org and ngx-01. We had issues the first 48 hours or so after the migration of both VMs to separate jails. This has drastically helped, though there are still latent issues.

Please let me know if you have trouble browsing our Radicle node[1]. We are still experiencing (much) higher than normal traffic, so due to our throttling, browsing our Radicle web interface may require hitting Refresh occasionally.

The next VM I plan to migrate to a jail is our rsync VM. I plan to do that after the next quarterly builds complete and are fully synced to our various mirrors. After rsync, I plan to modernize our Tor Onion Service endpoints.

I also enhanced our auto-sync program to support pushing to multiple remotes. In related news, syncing to GitHub is now supported once again. So if running a local Radicle node isn't your thing, you can reference our src and ports repos on GitHub. As of now, there are no plans to mirror to GitHub our auxiliary projects (hbsdmon, libhijack, vm-bhyve-hbsd, etc.

Given that the sync to GitHub happens when we perform our autosync (every six hours), there can be some delay between when direct commits land in the Radicle network and are subsequently mirrored to GitHub. My long-term plan is to write an orchestration daemon in Rust that communicates directly with the Radicle node via its control socket. It'll take action on the messages it sees on that socket. Think of it like "CI/CD lite"--something purpose-driven meant to satisfy HardenedBSD's needs. Also, I really enjoy reading Rust code, but I'm still not profficient in writing Rust, so I want to take this as an opportunity to better hone my Rust language writing skills.

Learning to write Rust better, anyways, will help me better submit patches to Radicle. There's still a decent amount of work needed. For example, it's possible to comment directly on a patch, but it's not possible to list patch comments (and hence reply to them.) Unless someone beats me to it (yes, please, if you have spare cycles), I hope to work on scratching that itch once I become more accustomed to writing Rust.

A note on the Radicle vulnerability announced[2] on 23 Sep 2026:

To sum up the vulnerability, there are two issues at stake:

  1. A malicious node can coerce another node to disclose private repositories (and their contents.)
  2. Radicle's traffic was thought to be fully encrypted. However, only the initial handshake is encrypted. The rest of the (long-lived) conversations betwen Radicle nodes is unencrypted.

HardenedBSD does not use private repositories on the Radcile network. We provide a Tor Onion Service endpoint for our main Radicle seed node. Using our Tor Onion Service endpoint ensures that your Radicle node is communicating through an encrypted transit. So we're not vulnerable to the private repo exposure issue. However, for those needing extra protection against traffic analysis, we suggest connecting to our Radicle node over Tor.

I still believe Radicle to be the right choice for HardenedBSD. However, their eventual migration to iroh will impact HardenedBSD, its users, and its developers. There will likely need to be coordinated effort between the Radicle team, the HardenedBSD team, and the wider community. I will be paying very close attention to Radicle's migration to iroh.

It is unfortunate that these kinds of vulnerabilities happened. Having worked on OpenSSL code, and integrated other projects in my past with OpenSSL, I can understand how something like this happened. Writing robust APIs is rather difficult.

At the same time, there should have been formal verification early on that things were working as desired. A simple tcpdump early on in development would have caught this. That said, I, too could have run tcpdump myself. However, I read over their documentation and took them at their word without verifying on my end. Same could be said for anyone and everyone even remotely interested in Radicle.

We're all human. We make mistakes. The Radicle team need to focus on adopting iroh, formalizing on-the-{,pseudo-}wire-traffic testing, and being more careful and focused with this next protocol iteration.

With all that said, I still want to be supportive of the Radicle team. They're already feeling pretty bad about making this level of magnitude mistake. They don't need anyone else piling on. Instead, they need our encouragement to get things right. Please be proactive in testing and, if you like writing Rust, help move them in that right direction.

[1]: https://radicle.network/nodes/rad.hardenedbsd.org
[2]: https://radicle.dev/2026/09/23/disclosure-of-vulnerability-in-network-pr...

Top

Kein stochastischer Papagei

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

„Ein LLM ist ein stochastischer Papagei“ heißt heute meistens: Der Prompt triggert Gelerntes, und das Modell gibt Trainingsdaten in anderer Form wieder. Es kann umformulieren, aber nichts Wesentliches abstrahieren und kein bekanntes Verfahren auf einen neuen Fall anwenden.

Das ist eine klare Behauptung über die Leistung des Modells. Sie ist falsch.

Der Ausdruck stammt aus On the Dangers of Stochastic Parrots von Bender und anderen . Dort bedeutet er etwas anderes: Sprachmodelle setzen sprachliche Formen nach statistischen Mustern zusammen, ohne selbst eine Mitteilungsabsicht oder einen Bezug zur Bedeutung der Worte zu haben.

Kurz: Das Modell erzeugt Sprache, aber es meint nichts damit.

Der Aufsatz warnt außerdem vor undokumentierten Trainingsdaten, übernommenen Vorurteilen, Umweltkosten und der Neigung von Menschen, plausiblen Text für eine Äußerung mit verantwortlichem Sprecher zu halten. Bender beschreibt selbst , wie sich die Verwendung des Ausdrucks vom Paper gelöst hat.

Ein Modell kann eine Aufgabe lösen, ohne dabei etwas mitteilen zu wollen. Darum entscheidet Benders These noch nicht, ob das Modell mehr als Gelerntes reproduzieren kann. Dafür brauchen wir einen Maßstab: Welche Arten von Aufgaben gibt es, und was muß bei einer gelungenen Lösung tatsächlich geleistet werden?

Vier Arten von Aufgaben

Der Deutsche Bildungsrat unterschied 1970 vier Lernzielstufen: Reproduktion, Reorganisation, Transfer und problemlösendes beziehungsweise entdeckendes Denken. Grob: Bekanntes wiedergeben; Bekanntes selbständig ordnen und verknüpfen; Gelerntes auf einen neuen Fall übertragen; für eine nicht routinemäßige Lage einen Lösungsweg entwickeln. Die deutschen Prüfungsanforderungen fassen das später zu drei Anforderungsbereichen zusammen: AFB I für Reproduktion, AFB II für Reorganisation und Transfer, AFB III für Reflexion, Problemlösung und Bewertung. Die KMK beschreibt diese Bereiche fachspezifisch; die Zuordnung ist kein mechanischer Wörterbuch-Lookup nach Aufgabenoperatoren.

Die deutsche Einteilung geht auf Arbeiten von Bloom aus dem Jahr 1956 zurück. Bloom ordnete Lernziele nach kognitiven Anforderungen. Die Revision von Anderson und Krathwohl von 2001 trennt zwei Dimensionen:

  • Was für Wissen? Faktenwissen, begriffliches Wissen, Verfahrenswissen oder metakognitives Wissen.
  • Was damit tun? Remember, Understand, Apply, Analyze, Evaluate oder Create.

Das ist ein wichtiger Unterschied. „Wende ein Verfahren an“ sagt noch nicht, ob Fakten, ein Begriffssystem oder ein Verfahren selbst Gegenstand der Aufgabe sind. Und eine Aufgabe kann mehrere dieser Prozesse verlangen.

Ein anderes Modell ist Webbs Depth of Knowledge . Es fragt nach der für die Lösung nötigen Denktiefe: DOK 1 ist Recall and Reproduction, DOK 2 Skills and Concepts, DOK 3 Strategic Thinking und DOK 4 Extended Thinking. Eine lange Aufgabe ist deshalb nicht automatisch DOK 4. Entscheidend ist, ob über mehrere Schritte Informationen integriert, Entscheidungen begründet und Erkenntnisse übertragen werden müssen.

Schließlich schaut die SOLO-Taxonomie von Biggs und Collis auf die Struktur der beobachteten Antwort: prestructural, unistructural, multistructural, relational und extended abstract. Nennt eine Antwort einen relevanten Aspekt, mehrere isolierte Aspekte, oder verbindet sie diese zu einem tragfähigen Ganzen? Kann sie das Ganze anschließend verallgemeinern?

Diese Systeme beschreiben Unterschiedliches. AFB, Bloom und DOK helfen, Anforderungen an eine Aufgabe zu charakterisieren. SOLO hilft, die Qualität einer konkreten Antwort zu beurteilen. Keines davon ist ein Intelligenzquotient für ein Modell.

Aufgabe AFB Revised Bloom ungefähr DOK
Definition wiedergeben I Remember 1
Sachverhalt in eigenen Worten erklären II, unterer Bereich Understand 1–2
Bekannte Informationen strukturieren II Understand/Analyze 2
Bekannte Methode auf neuen Fall anwenden II Apply 2–3
Unbekannten Fall und seine Beziehungen untersuchen II–III Analyze 3
Alternativen anhand von Kriterien beurteilen III Evaluate 3
Selbständig Modell oder Lösung entwickeln III Create 3–4
Über längere Zeit untersuchen und verallgemeinern III Analyze/Create 4

Das sind Beispiele, keine Umrechnungstabelle. Der konkrete Fall und die verlangte Begründung bestimmen das Niveau.

Von Reproduktion zu Transfer

Ein Modell soll „Hänsel und Gretel“ erzählen, aber die Kinder heißen Jens und Sophie. Das ist kein besonders starker Test. Namen in einer bekannten Geschichte kann auch ein triviales Programm ersetzen.

Interessanter wird es, wenn sich Rollen, Fähigkeiten und Schauplatz ändern. Welche Beziehungen tragen die Handlung? Welche Ereignisse müssen sich ändern, damit der Konflikt weiterhin funktioniert? Welche neue Variante bewahrt das Muster und welche zerstört es? Eine gute Antwort benutzt eine Abstraktion der Geschichte und überträgt sie auf den veränderten Fall. Das ist beobachtbar mehr als das Wiedergeben eines Dokuments.

Noch deutlicher ist Softwareentwicklung. Das Modell muß eine Anforderung als bestimmten Problemtyp erkennen, relevante Dateien finden, mögliche Verfahren auswählen und an der konkreten Codebasis anwenden. Bei einer Fehlersuche muß es Zusammenhänge zwischen Aufrufstellen, Datenfluß und beobachtetem Fehler herstellen. Das sind Leistungen im Bereich Apply und Analyze. Die schlichte Klassifikation eines Falls gehört bei Revised Bloom allerdings zunächst zu Understand. Erst das Zerlegen, Unterscheiden und Begründen der Beziehungen macht daraus Analyze.

Das ist die Stelle, an der der verbreitete Papageienvorwurf scheitert. Eine Antwort kann Beziehungen zwischen bekannten Elementen erkennen und auf einen neuen Fall übertragen. Ob dabei menschliches Verstehen stattfindet, ist eine andere Frage.

Segmentieren und prüfen

Ein einzelner Prompt ist ein schlechter Arbeitsprozeß für ein großes Problem. Man muß die Arbeit zerlegen: Ziel klären, unbekannte Voraussetzungen prüfen, Teilaufgaben bestimmen, Ergebnisse jeweils abnehmen und erst dann den nächsten Schritt beginnen. Das ist für Menschen normale Projektarbeit. Für LLMs ist es besonders wichtig, weil Kontext begrenzt ist und ein früher Fehler sonst in allen späteren Antworten weiterlebt.

Bei der Codegenerierung kann ein Agentic Harness diese Arbeit unterstützen. Es gibt dem Modell Zugriff auf Repository, Dateien, Shell, Git und Tests. Entscheidungen liegen als Dateien vor und überleben einen neuen Modellkontext. Ein Ticket kann klein genug werden, daß sein Ergebnis prüfbar ist. Ich habe diesen Ablauf in „LLM Driven Development“ und am konkreten Projekt in „Building server-key-injection by specialization“ beschrieben. „Die Menschmaschine muß eingearbeitet werden“ erklärt, weshalb lokales Firmenwissen und Einarbeitung dabei entscheidend sind.

Mechanische Quality Gates machen einen Teil der Arbeit überprüfbar:

uv run pytest
uv run ruff check
uv run ty check

Ein Test kann beobachtbares Verhalten prüfen. Ein Linter kann Regelverstöße finden. Ein Typechecker kann bestimmte Widersprüche im Code aufdecken. Git macht Änderungen sichtbar und rücknehmbar. Das Modell kann Fehlermeldungen lesen, eine Hypothese korrigieren und erneut prüfen. Es bleibt nicht bei einer einzigen plausiblen Antwort.

Grüne Gates beweisen allerdings nur, was sie tatsächlich prüfen. Ein Test, der bloß einen Mock-Aufruf wiederholt, sagt wenig über das Verhalten des Programms. Ein Typechecker kennt die Geschäftsanforderung nicht. Ein Benchmark ist ebenfalls ein von außen angelegtes Gate; sein Bestehen macht die Bewertung nicht automatisch zu einer eigenen Evaluate-Leistung des Modells. Evaluate zeigt sich, wenn es die Prüfkriterien hinterfragt, Lücken erkennt und eine aussagekräftigere Prüfung entwirft.

Die Leistung gehört dem System aus Modell, Harness, Kontext, Werkzeugen und menschlicher Führung. Wie groß der Anteil der Umgebung ist, zeigt auch die Forschung zu SWE-agent : Schon die Gestaltung der Schnittstelle zum Computer verändert die Ergebnisse bei Softwareaufgaben erheblich.

Das ist kein Einwand gegen die Leistung. Ein Mensch löst Softwareprobleme ebenfalls mit Editor, Dokumentation, Tests und Kollegen und typischerweise nicht in einem One-Shot. Man muß nur sauber benennen, was man gemessen hat. Ein Agent mit Tests ist nicht dasselbe Versuchsobjekt wie ein nackter Modellaufruf.

Modelle sind keine Anforderungsbereiche

Luna, Terra, Sol und Astra sind keine Stufen von Bloom. Die aktuelle OpenAI-Modellübersicht führt GPT-6 Luna für eng umrissene Arbeit in hoher Stückzahl, GPT-6 Sol für anspruchsvollere Arbeitsabläufe und GPT-6 Astra für besonders schwierige Aufgaben.

Eine präzise spezifizierte AFB-III-Aufgabe kann ein günstiges Modell mit guten Gates lösen. Ein starkes Modell kann an einer AFB-I-Frage scheitern. Für Architektur und unklare Anforderungen ist mehr Urteilskraft wertvoll; nach der Segmentierung können kleine Modelle enge Tickets abarbeiten. So war es auch im verlinkten server-key-injection-Versuch. Die Taxonomie beschreibt die Aufgabe und die Antwort. Modellwahl ist eine Frage von Erfolgsrate, Kosten und benötigter Führung im konkreten Prozeß.

Verstehen und Erzeugen

Die Taxonomien liefern keine einfache Antwort auf die Frage, ob ein LLM versteht. Sie klassifizieren Anforderungen und sichtbare Leistungen, nicht subjektive Erfahrung oder eine bestimmte innere Repräsentation. Wenn ein Modell einen fremden Fehler richtig einordnet und behebt, darf man die funktionale Leistung beschreiben. Daraus folgt noch keine Gleichheit mit menschlichem Verstehen. Umgekehrt verschwindet die Leistung nicht, nur weil das Modell anders arbeitet als ein Mensch.

Bei Create ist die Grenze noch interessanter. Ein neues Stück Code ist nicht schon Forschung. Es kann eine Variante einer bekannten Lösung sein. Eine neue Hypothese ist ebenfalls noch keine Erkenntnis. Sie muß etwas riskieren: Sie muß an einem bislang nicht verwendeten Fall scheitern können.

Es gibt bereits Beispiele, in denen LLMs an neuen Ergebnissen beteiligt waren. FunSearch kombinierte ein Sprachmodell mit einer Suche und einem automatischen Evaluator und fand neue mathematische Konstruktionen sowie Algorithmen. Das ist eine Leistung des zusammengesetzten Systems unter günstigen Bedingungen: Das Problem ließ sich so formulieren, daß Kandidaten automatisch bewertet werden konnten. Daraus folgt nicht, daß jedes Modell allein ein offenes Forschungsprogramm führen kann.

Wissenschaft als Quality Gate

Murray Gell-Mann verweist in The Quark and the Jaguar , Kapitel 7, auf Popper: Eine wissenschaftliche Theorie macht Vorhersagen, an denen sie scheitern kann. Wiederholte Beobachtungen gegen diese Vorhersagen widerlegen sie. Das ist das erste Quality Gate für ein neues Modell. Vor dem Test müssen Vorhersage, Gültigkeitsbereich und nötige Genauigkeit feststehen. Dann prüft man an Beobachtungen, die nicht schon zur Konstruktion des Modells benutzt wurden.

Das zweite Gate fragt nach der funktionalen Erklärung: Welche Beziehungen und Mechanismen erzeugen das beobachtete Verhalten? Was sollte sich ändern, wenn man gezielt in das System eingreift? Gell-Mann beschreibt in Kapitel 9 die Suche nach solchen Mechanismen, warnt aber vor einer vorschnellen Abwertung beschreibender Theorien. Darwin konnte die Vererbung und Entstehung von Variation noch nicht erklären. Seine Evolutionstheorie war deswegen nicht wertlos. Das zweite Gate vertieft ein brauchbares Modell; es ersetzt das erste nicht.

Newtons Gravitation sagt Bewegungen in ihrem Gültigkeitsbereich voraus. Einstein erklärt Gravitation geometrisch; bei schwachen Feldern und langsamen Bewegungen ergibt sich wieder Newtons Beschreibung. Welche Rolle Gravitation in einer Quantentheorie spielt, bleibt offen. Erklärung hat Ebenen, und keine davon erspart den Vergleich mit der Beobachtung. Zum Newtonschen Grenzfall: MIT OpenCourseWare .

Dasselbe Maß muß für menschliche und maschinelle Vorschläge gelten. Wer eine Theorie formuliert hat, ist eine Frage. Ob sie neue Fälle vorhersagt und Zusammenhänge erklärt, ist eine andere. Menschen bleiben bei Zielwahl, Versuchsplanung, Datenqualität und Interpretation in der Verantwortung. Ein LLM kann bei allen diesen Schritten helfen; ob es ein bestimmtes Modell erfolgreich gebildet hat, entscheidet sich an den beiden Gates und nicht an der Eleganz seiner Antwort.

Benders Frage bleibt wichtig: Eine plausible Antwort hat keinen menschlichen Sprecher, der mit ihr etwas beabsichtigt und die Folgen in der Welt erlebt. Das sagt noch nicht, welche Aufgaben das System lösen kann.

TL;DR

Die heute übliche Behauptung lautet: Ein LLM gibt Gelerntes in wechselnden Formulierungen wieder, kann aber nichts Wesentliches abstrahieren oder auf einen neuen Fall anwenden.

Diese Behauptung ist nicht nur falsch, sie macht aus einer Diskussion über die Stärken und Schwächen der KI ein fast religiöses Dogma und verstellt den Blick auf die wichtigen Fragestellungen.

Mit einem geeigneten Prozeß leisten LLMs bei klar abgegrenzten Codeaufgaben zuverlässig mehr als Reproduktion: Sie ordnen Informationen neu, übertragen Verfahren, analysieren Probleme und verbessern Lösungen anhand überprüfbarer Ergebnisse. Ob daraus menschliches Verstehen folgt und wie weit es bei eigenständiger Forschung und Modellbildung trägt, entscheidet dieser Befund nicht. Dort müssen neue Vorhersagen und funktionale Erklärungen die nächsten Quality Gates sein.

Top

Why just switcharoo'ing RAM on a RPI isn't guaranteed to work, or "wait, you have to do WHAT to get DDR to work?"

Post by Adrian Chadd via Adrian Chadd's Ramblings »

There's been some fun stuff lately in the maker community around Raspberry Pi and RAM upgrades. Specifically, a recentish (~ 2025) boot firmware upgrade locks the RAM size to what it shipped with. There's been some write-ups about it - https://www.jeffgeerling.com/blog/2026/raspberry-pi-ram-lockdown/ and https://hackaday.com/2026/09/22/raspberry-pi-ram-restrictions-no-big-deal-frankly/ come to mind - but they don't REALLY go into all the gory details behind the scenes.

So let's do a quick dive into RAM and how it's changed from the past until now.

First off - a lot of tinkering in retro computing exposes people to static and dynamic RAM chips, first dating back to the late 70s. After a few attempts at stuff, manufacturers soon ended up kind of iterating towards a few standard pinouts and specification timings - likely so new upstart RAM makers could sell their chips as "drop in replacements" for existing RAM.

This led manufacturers towards defining an actual standard for this stuff - JEDEC (https://en.wikipedia.org/wiki/JEDEC_memory_standards) - which defines what and how various types of RAM need to behave. This is needed so you didn't end up with RAM chips needing to be paired with very specific controllers / CPUs. Some vendors kept the market on their toes with very not compatible stuff (think the RDRAM from Rambus - https://en.wikipedia.org/wiki/RDRAM) but in general one of the big reasons we had affordable RAM until recently (and will again, I'm sure) is because there's basically no vendor lock-in and a lot of interoperability.

However, that interoperability comes at a cost. Silicon isn't just some magical thing you can stamp out and get a million chips that all behave exactly the same. There's always going to be subtle behavioural differences across them. For CPUs this can actually show up as different maximum clock frequency versus heat dissipation - some CPUs will run at the maximum clock frequency at a lower temperature versus others - and yes when you graph them you'll likely get a normal or bi-modal distribution. For desktop CPUs with active cooling this may result in the same CPU being sold at a higher price with a higher maximum clock frequency, and then a "slightly cheaper" CPU being sold with a lower maximum clock frequency. If it's going into something like a cellphone or a piece of wearable technology where there's no active cooling, you either have to choose whether to lower EVERYONES maximum device performance, allow SOME devices to operate at a maximum higher performance, or you reject a percentage of the CPUs from the product and use them elsewhere.

(Yes, this is what they call "binning".)

For RAM, stable frequency, timing and bus characteristics are critical. Remember these parts aren't operating at 1 or 10MHz anymore - these chips operate their busses on both sides of the clock cycle at over 1GHz. At that point you're well into the "this behaves like RF" domain. As part of this, the PCB layout design, the connector design, the individual track lengths (think phase delays) + lengths inside the memory controller all have to line up exactly correctly - and then you still have weird RF style behaviour going on. Remember how I said every chip behaves differently? They're going to have slightly different timing and slightly different RF characteristics. So you can't just whack a single set of parameters down and hope they work - you need to go through calibration, validation and bulk testing before you know you've got something that works.

First up - the DDR4 power-up sequence is not just "apply power, add RAS/CAS timings and go." There's a whole calibration sequence that has to happen to calibrate the memory controller PHY (physical interface). For an example of this, take a look at the AMD Soft core (FPGA) DDR4 SDRAM controller calibration / setup documentation at https://docs.amd.com/r/en-US/pg353-versal-acap-soft-ddr4-mem-ip/Memory-Initialization-and-Calibration-Sequence . It has to be able to run this sequence to completion, every time, or you won't have reliably working RAM. 

What parameters are required to make this work?

Well, we get cheap, mostly interoperable DIMMs because the JEDEC standard defines what parameters to put into the DIMM EEPROM module that is read by the memory controller during power on. Yes, it's also documented online - https://en.wikipedia.org/wiki/Serial_presence_detect has all the fun details. Note that there's many, many more parameters here than just RAS/CAS timings! There's all kinds of timings to do with voltage calibration management, nanosecond accurate wait and hold times for various parts of the RAS/CAS signaling, data line skew, and other fun behavioural things.

Now here comes the fun part.

A DIMM manufacturer is going to make some test DIMMs and test them on a whole variety of motherboards to make sure that the parameters they program in reliably result in the above PHY calibration/setup succeeding 100% of the time on all the hardware they care about. They're then going to choose (hopefully conservatively) suitable values to write into the SPD to allow this to, yes, calibrate/setup 100% of the time. They may even slow down the RAM to meet timing if needed. But typically they're going to hope that the manufacturer binning of the SDRAM parts into specific part numbers and performance/behavioural bands will result in being able to make a DIMM that consistently operates at a specific performance tier without them having to further bin things when they're on the DIMM. Noone wants to throw away manufactured things, so the manufacturer has the option to just write slower parameters into SPD and sell the DIMM as a slower DIMM.

But an embedded board developer doesn't have some EEPROM attached to each group of RAM modules they've put on the board. They will have likely characterised one, two or three RAM module suppliers (either for different sizes, or the same size but multiple manufacturers so they can play off pricing against each other when making different batches of boards) and put those parameters into a fixed lookup table on said embedded board as a starting point. If you're fancy (and I don't know if RPI is being fancy or not) you'll actually run the calibration on your manufacturing line and write the SDRAM parameters for your chosen RAM into the EEPROM as the "active" configuration.

And here you see the dilemma.

See, the configuration parameters are either conservative ones for a handful of tested RAM chips from specific manufacturers, or they're very specifically for the RAM you've had installed on your embedded board. They've done a whole lot of characterization of the RAM across first one, then three, then a dozen or two, then hundreds, and finally the whole run as they're making them, to minimise the chances that during actual bulk manufacturing that a board fails RAM calibration/setup.

So when you just switch the SDRAM chip over to some other manufacturer part? You're basically hoping that given the initial RAM configuration parameters stored in EEPROM/flash somewhere on your board, the RAM calibration/setup will always succeed and then normal operation is stable. And you may get lucky! You may not! But you absolutely aren't guaranteed anything at this point.

And this is why the Raspberry Pi Foundation is doing the right thing here. Sorry y'all, but at this point you're actually likely not qualified to do all of the above work.

... and why not? Why can't you calibrate/test the RAM and write the parameters back?

A great question! The short answer is this.

It's typically vendor provided tooling that is not open source and is very proprietary, and they don't want you to know how to do it or what their memory controller is doing.

And THAT is a discussion for another time.

So now you know. It's not as simple as "RPI foundation is good" or "RPI foundation is bad". It's an intersection of how modern hardware works, how much effort goes into making sure it's bullet proof (or as bullet proof as they can get it) and what vendors / manufacturing lines are willing to release versus what they're requiring to keep as closed / NDAed stuff.

 

(Don't hate the player, hate the game, etc) 

Top

U.S. Soldier Gets 70 Months in Prison for AT&T, Verizon Extortions

Post by Brian Krebs via Krebs on Security »

A U.S. Army soldier who pleaded guilty to hacking into multiple telecommunications companies and stealing mobile call and text metadata for more than 100 million AT&T customers in 2024 was sentenced to 70 months in federal prison today and ordered to pay nearly $300,000 in restitution to victims.

One of several selfies from the Facebook page of Cameron Wagenius.

Cameron John Wagenius, 22, was stationed at a U.S. Army base in South Korea when he adopted the cybercriminal persona “Kiberphant0m.” Working with three alleged co-conspirators, Kiberphant0m downloaded data from several large customers of the cloud data storage service Snowflake that had exposed credentials and did not enforce multi-factor authentication (Snowflake has since mandated MFA on all accounts).

In October 2024, Kiberphant0m bragged on the cybercrime forums that he’d stolen the call and text metadata (e.g. source and destination number, timestamp, duration, etc.) for tens of millions of AT&T customers. Kiberphant0m claimed to have hacked into more than dozen telecommunications companies worldwide, including Verizon’s Push-to-Talk business, and publicly extorted these companies in exchange for a promise not to publish the stolen data.

In late November 2025, KrebsOnSecurity warned that Kiberphant0m was likely a U.S. soldier stationed in South Korea. Less than a month later, Wagenius was arrested and charged in two separate federal indictments, and soon pleaded guilty to all counts in both cases.

At his sentencing hearing in Seattle today, Wagenius was sentenced to nearly six years in federal prison, and ordered to pay $294,978 in restitution.

Federal prosecutors said Wagenius was assisted in his efforts to extort victim companies by Kenneth Schuchman, a 28-year old man from Vancouver, Washington who has a lengthy cybercriminal history. In 2019, Schuchman pleaded guilty to operating the Satori botnet, a vast collection of hacked Internet-of-Things (IoT) devices that was used for large-scale distributed denial-of-service (DDoS) attacks.

Two other alleged co-conspirators of Wagenius are still facing charges in connection with the Snowflake data thefts; Conor Riley Moucka, a.k.a. “Judische,” of Kitchener, Ontario was arrested in 2024 and pleaded guilty in August 2026; and John Erin Binns, an American man currently living in Turkey who is also wanted for a 2021 data breach at T-Mobile that exposed the personal information of at least 76 million customers.

Kiberphant0m also admitted to re-extorting victims, and threatening to disclose national security secrets. Immediately following Moucka’s arrest — after AT&T had already paid the extortion group a $370,000 Bitcoin ransom — Kiberphant0m posted on hacker forums what he claimed were the AT&T call logs for then President-elect Donald Trump and for then Vice President Kamala Harris, as well as schematics allegedly stolen from the U.S. National Security Agency (NSA).

Paul Russell is a resident agent in charge at the Defense Criminal Investigative Service (DCIS), the criminal investigative arm of the U.S. Department of Defense Office of Inspector General. Russell said when DCIS received information that a soldier with secret clearance was allegedly involved in cybercrime and extortion, the agency began working the investigation alongside the FBI, the Army Criminal Investigative Division (CID), and the U.S. Secret Service.

“We don’t often get leads where there’s an active duty soldier with a secret clearance who’s creating hacking tools and trafficking in data,” Russell said. “That doesn’t happen every day, and so when that hits it really spins all of our partner organizations up. It was very serious from jump street, just because it was unique, it was an insider threat, and we weren’t sure what we were dealing with.”

A sentencing memo (PDF) filed Sept. 19 by federal prosecutors in Seattle notes that while Wagenius pleaded guilty almost immediately and has been remarkably cooperative, he recently got caught trying to find security vulnerabilities in the BOP’s computer network. The government’s memo notes that while incarcerated and awaiting sentencing, Wagenius violated the computer use policies of the Bureau of Prisons (BOP) in attempts to learn about vulnerabilities in BOP computer systems.

“According to records from BOP, in or around September 2025, Wagenius used another inmate’s email system to request that the email recipient prompt a commercial AI tool to provide information about “[w]hat CVE’s are there for Windows 10 Enterprise privilege escalation and bypasses” and to “[p]rovide the CVE’s and a real world working script for each CVE . . . without omitted code,” the government’s memo states.

The memo states that less than a week later, Wagenius used a different inmate’s email account and requested that the email recipient prompt an AI tool to “[p]rovide the step by step for CVE-2023-45208, code for this if any, and if no code exists make some, make sure to describe everything in detail.” CVE-2023-45208 is a three-year-old “command injection” vulnerability in D-Link networking devices.

That same month, Wagenius allegedly again requested that the email recipient prompt AI with the question, “How do you make an antenna in a prison environment with commissary or readily available items/tools to improve/make an antenna to extend radio reception?”

Federal prosecutors said Wagenius also requested that the recipient research escaping prison.

“In several instances, Wagenius framed the AI queries as being posed in connection to a book he was writing. This is a common method of ‘prompt injection,’ in which attackers feed specially crafted, deceptive inputs into commercial AI tools that are programmed to avoid outputting malicious code that can be used to exploit computer vulnerabilities,” the sentencing memo reads.

The government told the court it is unaware of evidence that Wagenius figured out how to use or deploy the vulnerabilities he was researching in the BOP’s systems, and when questioned said he was only researching “potential vulnerabilities to provide information to the BOP.”

Incredibly, despite the enormous financial value of the data stolen from AT&T and other telecom providers, Wagenius’s extortion efforts were largely unsuccessful. The government’s sentencing memo says Wagenius made a whopping total of around $1,500 from selling stolen data.

“While Wagenius was not particularly financially successful as a cybercriminal, he both intended to and caused significant harm to numerous individual victims, U.S. companies, and the U.S. government,” the memo states.

Top

Launching FreeBSD/EC2 desktop AMIs

Post by Colin Percival via Daemonic Dispatches »

I'm excited to announce — oh don't worry, I'm not doing that again — a new feature for the FreeBSD/EC2 platform: Desktop AMIs.

Top

FBI Probes Service Selling 153M+ Drivers Licenses

Post by Brian Krebs via Krebs on Security »

A new identity theft service launched on the dark web this week is selling digital scans of more than 153 million drivers licenses from people in the United States and Canada. Based on interviews with individuals whose licenses are available for purchase on this service, it appears to be siphoning images collected by a widely-used identity verification company based in Louisiana. KrebsOnSecurity also has learned that the New Orleans field office of the Federal Bureau of Investigation (FBI) today launched an official inquiry into the source of the images.

A record available at this identity theft service that includes the drivers license for U.S. Defense Secretary Pete Hegseth, one of several high-ranking U.S. government officials whose drivers licenses can be found for sale.

On Monday, Aug. 31, a source alerted KrebsOnSecurity to a service advertised by a new user on the Russian cybercrime forum Exploit, offering access to digital scans of identity documents on more than 170 million people in North America. The source brought it to my attention because the proprietor of this identity theft service offered my Virginia drivers license as a free sample in their initial sales thread on Exploit.

The service, dubbed Nexus, claims to have more than 153 million drivers licenses for people in the United States and Canada, as well as more than 10 million identification cards; more than three million travel documents and/or international IDs; and at least 579,000 medical cards.

A quick look around Nexus finds they are likely not exaggerating about that 153 million number: Running a blank search in Nexus (with no search parameters entered) returns approximately 11.5 million pages of results, with roughly 15 results displayed per page. It includes documents from people in both Canada and the United States, but the bulk of these records are on Americans: searching for just Canadian drivers licenses returns approximately 1.1 million results, with the largest concentration from Ontario (473,673 records).

Curiously, the identity records include not only drivers licenses but also marijuana dispensary cards. Some of the records list their “source” as “CDL,” presumably short for “commercial drivers license.” Other records carry the source notation of “CAC,” which may refer to Common Access Cards, government issued identity cards that grant physical access to government buildings and secure rooms.

The people behind Nexus claim the license images are coming from an active breach at “a major identity verification company” whose customers include multiple Fortune 500 companies.

The record totals listed by the Nexus identity theft service. The number of drivers license records increased by nearly 400,000 in the span of just 24 hours.

“We have been continuously exfiltrating new data for over a year into our private database,” the service enthused in its introductory post on Exploit. “Records are available to preview before purchase with pertinent information redacted. Customer photos are displayed if available.”

Indeed, over the past 24 hours, the number of drivers license records listed as available in Nexus has increased by nearly 400,000, suggesting that freshly stolen license data is being harvested and uploaded to this service on a semi-regular basis.

The record featuring my drivers license includes six image files: three pairs of photos of the license’s front and back, a basic image scan, as well as infrared and ultraviolet versions of the same images. A date and timestamp is appended to each image file, and the timestamp on my license scan corresponds to a date in June 2025 when I took a flight to the midwest United States to attend a family funeral.

Some of the 153 million+ license scans — including mine — feature six image files with date and timestamps appended to the filenames. Not all records include photos, and some that do feature photos do not display the associated filenames.

Intent on discovering the source of this data, KrebsOnSecurity asked more than a dozen friends and family members for permission to search for their licenses in this service. Each person whose license could be found (nine of them) confirmed having traveled on or very close to the dates in the timestamps attached to their images. It is unclear what timezone these timestamps are in, but from reviewing car rental records shared by several people who helped with this research, it appears the timezone is set to Greenwich Mean Time (GMT).

At first, I thought the source of the data might have something to do with airports. However, that theory went out the window when it became apparent there were no passports in this data set. Also, only some of those who helped with this research said they showed their drivers license at the airport on the day of their travel. One person whose license was in Nexus hadn’t flown at all recently, but was renting a car from Hertz for several months around the date of their timestamp.

Two of those who agreed to help are federal employees who said they shared other forms of government identification when passing through airport security. However, those individuals each said they shared their state-issued drivers licenses later that day when renting vehicles at their respective destinations, and that both rented their cars from Hertz.

After finding a note in my calendar for the day of my June 2025 flight reminding me to bring my passport, I remembered that I also never actually shared my drivers license when I went through security at Reagan National Airport on that day because I did not yet have a Real ID, a security-enhanced drivers license that is now required by the Transportation Security Administration (TSA) for all domestic travel. Instead, I showed the TSA agent my government-issued U.S. passport.

Here’s where it gets interesting: I was able to find my mother’s drivers license in this service as well, and the timestamps for her images are just a few seconds apart from mine. That’s notable because we both handed our licenses to the Hertz rental car representative at the same time.

According to my mom, the only place she gave her drivers license to that day was the rental car company, and if memory serves that is also true for me. I don’t recall if the rental car representative inserted our licenses into any kind of machine, but I remember they held onto them for several minutes behind the counter while we were signing various forms. KrebsOnSecurity sought comment from Hertz and will update this story in the event they reply.

Zach Edwards is a well-known security and privacy researcher who recently launched a service called DecryptAds to help people better understand how online advertisers are tracking them. A scan of Edwards’s drivers license is available for purchase on this identity theft service, and Edwards said the timestamp on his record corresponds to the middle of a trip last month to Las Vegas for the annual DEFCON security conference.

Edwards told KrebsOnSecurity that although he did not rent a car in Vegas, he did hand over his license at the TSA checkpoint, at a marijuana dispensary in Vegas, and at his hotel (the Aria). But he said the only one of those three that for sure scanned his ID in some kind of device was the dispensary.

To enter Planet13’s weed dispensary in Las Vegas, one must pass through a red telephone booth. Image: Zach Edwards.

Edwards said the dispensary he visited that day was Planet13, a multi-state chain with stores in California, Florida, Illinois and Nevada. In 2022, the New Orleans-based identity provider idscan.net published a press release announcing an exclusive identity verification agreement with Planet13’s dispensaries nationally. IDScan says it processes ID verification for more than 1,000 marijuana dispensaries in 19 U.S. states.

The “trust” page of idscan.net states that the company provides identity verification services for numerous big brands, including Hertz, Target, Fedex, Motorola Solutions, the financial services giant Jack Henry, and Caesars Entertainment. And as idscan.net’s own documentation states, the technology scans IDs with both infrared and ultraviolet light. Idscan.net says the company’s systems and technology perform more than 21 million verifications monthly, at more than 20,000 locations around the world.

Image: idscan.net.

Contacted by KrebsOnSecurity, idscan.net said it was investigating the matter, but the company has not yet shared an official statement or a substantive reply to specific questions sent via email.

“At this point I’m not able to share any additional information, but the updates you have provided have been welcome, and helpful to our team’s investigation,” wrote Jillian Kossman, a marketing and operations leader at idscan.net.

During the course of my research for this story, word got around to the FBI that I was poking at the apparent source of this new identity theft service’s data. Probably they were tipped off when I shared with a trusted source that Nexus also is selling the drivers license information for the assistant director of the FBI (I did not find FBI Director Kash Patel’s license in Nexus).

Earlier this afternoon, I was added to a conference call with a half-dozen FBI agents, including senior leaders from the agency’s cyber division. During that call, the FBI shared that earlier today their New Orleans field office opened an official investigation into an apparent breach involving idscan.net.

Edwards said that as more in-person and online experiences require sharing drivers licenses, vendors who collect this sensitive data need to be held to a higher standard.

“This episode should further strengthen the resolve for people who are fighting back against online ID schemes which are requiring countless providers to ask for drivers licenses in order to access services under the guise of protecting kids,” Edwards told KrebsOnSecurity. “These systems are putting sensitive data into more and more 3rd party vendors, and we don’t have nearly the oversight to ensure they are safe.”

Larry Baldwin is principal intelligence researcher at the cybersecurity firm Cybera. Baldwin said a front and back scan of his drivers license available at Nexus contains timestamps that correspond to the date of a car rental from Hertz on a recent vacation.

Baldwin said the Nexus identity theft service presents multiple serious security and privacy threats, noting that state-issued drivers licenses are commonly used as proof of one’s identity when opening new lines of credit. Baldwin said the service could also dangerously expose many people who do not wish to be found but who cannot meaningfully change their appearance (or at least not enough to fool today’s AI-based image matching tools).

This category of people, he said, includes those fleeing domestic violence, and even people who have been assigned a whole new life and identity as part of the federal government’s witness protection program, which is generally reserved for criminal defendants in racketeering and conspiracy investigations who agree to cooperate with federal authorities.

“Just when it seems like we’re making some headway in improving authentication controls through drivers license verification systems, this happens and the very thing those improvements are dependent on are compromised,” Baldwin said.

Update, Sept. 8: IDscan.net published a brief notice saying it has “determined that an unauthorized third party may have accessed and/or copied certain customer information, including full names and drivers license or other government-issued identification numbers.” The statement said IDscan.net is notifying affected individuals and offering credit protection services.

Update, Sept. 2, 6:05 p.m. ET: A spokesperson for Caesars Entertainment said Caesars has not been a client of IDScan.net and has not used VeriScan since February 2025, despite IDScan.net listing them as a client on their website. That person said Caesars had no active VeriScan accounts at the time of the incident and did not authorize IDScan.net to retain data from its accounts, and that IDScan.net said the incident should have no impact on Caesars Entertainment.

Update, 8:56 p.m. ET: Shortly after this story was published, the Nexus identity theft service website vanished from the darkweb, replacing its login page with a plain text message that reads, “This service is no longer available.”

This is a potentially fast-moving story. Any changes or updates will be noted here along with a timestamp.

Top

Expat 2.8.5 released, fixes vulnerability CVE-2026-93990

Post by Sebastian Pipping via Hartwork Blog »

For readers new to Expat:

libexpat is a fast streaming XML parser. Alongside libxml2, Expat is one of the most widely used software libre XML parsers written in C, specifically C99. It is cross-platform and licensed under the MIT license.

Expat 2.8.5 was released a few hours ago. The key motivation for cutting a release and doing so now was to get the fix for vulnerability CVE-2026-93990 into the hands of the users.

The vulnerability was reported and fixed by Kartik Kenchi — thank you! Expat prior to release 2.8.5 did not stop attackers from smuggling malformed UTF-16 into the application using Expat, where it could cause arbitrary damage, depending on how malformed UTF-16 was handled inside the application. The issue was that surrogate pairs were not validated for the second half being a "low surrogate" in the range of 0xDC00 to 0xDFFF.

As usual, there is also non-security work that went into this release: bugfixes and improvements to the xmlwf command line utility, improvements to the two build systems, the documentation, the infrastructure, and also internals. For more details about this release, please check out the change log.

It should be noted that it was the Open Source Sabbatical of the City of Munich that allowed me to focus on libexpat for this release through its funding — thank you! We're in the second month of hopefully six months total; there continues to be plenty to do. Wish me luck!

Thanks to everyone who contributed to this release of Expat!

If you maintain Expat packaging, a bundled copy of Expat, or a pinned version of Expat, please update to version 2.8.5. Thank you!

Sebastian Pipping

Top

Valuable News – 2026/09/21

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

Standalone C Memory Treemap Program for FreeBSD.
https://github.com/johnrobdoyle/FreeBSD-Memory-Treemap

Closing IPv6 First Packet Gap with GRAND in FreeBSD.
https://labs.ripe.net/author/pouria/closing-the-ipv6-first-packet-gap-with-grand/

Watchgoose Allows Cron Job Monitoring and Heartbeat Alerts.
https://watchgoose.com/

FreeBSD Git Weekly: 2026-09-07 to 2026-09-13.
https://freebsd-git-weekly.tarsnap.net/2026-09-07.html

KeePass Compatible Native Password Manager Kura for Haiku OS.
https://github.com/ilfelice/kura

Learning at Scale: FreeBSD Foundation Intern Nimish Jain on Exploring FreeBSD Codebase.
https://freebsdfoundation.org/blog/learning-at-scale-freebsd-foundation-intern-nimish-jain-on-exploring-the-freebsd-codebase/

GEFS Running on OpenBSD.
https://marc.info/?l=openbsd-tech&m=178948744271633&w=2

GEFS Good Enough File System.
https://orib.dev/gefs.pdf

FreeBSD: Forgotten Giant of Modern Infrastructure.
https://harriszinsouvi.substack.com/p/freebsd-the-forgotten-giant-of-modern

MariAdmin and OpenBSD on OVHcloud and jstats(1) Updates.
https://ozgurkazancci.com/long-time-no-text-jstats-updates-mariadmin-and-openbsd-on-ovhcloud/

Run GitHub CI in FreeBSD.
https://github.com/vmactions/freebsd-vm

Hidden Cost of Upgrading FreeBSD in Production.
https://klarasystems.com/articles/freebsd-the-hidden-cost-of-upgrading-in-production/

MidnightBSD arctic_fan Driver for ARCTIC Fan Controller.
https://github.com/laffer1/arctic_fan_driver

FreeBSD 16 on Framework Laptop – One Day – Full Desktop.
https://linkedin.com/pulse/freebsd-16-framework-laptop-one-day-full-desktop-devin-teske-rbdac/

Basic Remote Filesystem for 2.11BSD UNIX.
https://github.com/DoctorWkt/211BSD_Brf/

NetBSD on My Netbook.
https://movq.de/blog/postings/2026-09-18/0/POSTING-en.html

Apiary – FreeBSD Bhyve/Jails Native Cluster Virtualization Management Platform.
https://github.com/glenjbarber/apiary

Does Anyone Also Know the Daemonettes?
https://reddit.com/r/freebsd/comments/1v3vukz/

FreeBSD Bhyve (Sylve Specific) Prometheus Exporter Written in Go.
https://github.com/gyptazy/sylve-bhyve-exporter

GSoC 2026 Reports: Port Enlightenment Desktop Environment to NetBSD – Part 2.
https://blog.netbsd.org/tnf/entry/gsoc2026_enlightenment_2

WebZFS.
https://alfaexploit.com/en/posts/webzfs/

UNIX/Audio/Video

EuroBSDCon 2026 Videos/Presentations.
https://exquisite.tube/c/ebc2026/videos

EuroBSDCon 2026 OpenBSD Related Talks:
https://undeadly.org/cgi?action=article;sid=20260916093106

Debug Program Starts with extrace(1) on FreeBSD.
https://youtube.com/watch?v=_Mfg-z9OK6I

FreeBSD Rants: Why Do I Use FreeBSD?
https://youtube.com/watch?v=ESSyz3KUMDw

FreeBSD Basics – Part 4 – System /bin/sh Shell.
https://youtube.com/watch?v=ofUcRwUCG5I

FreeBSD Basics – Part 5 – System /bin/sh Shell.
https://youtube.com/watch?v=jCt3UUJ_J1g

2026/09 FreeBSD Developer Summit.
https://youtube.com/watch?v=L_xAGzlSkvs

2026-09-17 bhyve Production User Call.
https://youtube.com/watch?v=g333arHWYmM

2026-09-15 Jail/Zones Production User Call.
https://youtube.com/watch?v=IcjkGCvlQB8

GhostBSD Native Echo Port.
https://youtube.com/watch?v=1oWf8cnwRgU

BSD Now 681: UNIX Still Powering World Infrastructure.
https://www.bsdnow.tv/681

Hardware

AMD Releases New Ryzen 5 5500F and 7500F Budget CPUs for AM4 Socket.
https://hwcooling.net/en/amd-releases-new-budget-cpus-even-for-the-am4-socket/

Joy of Making Stuff with 3D Printer.
https://notetoself.studio/post/the-joy-of-3d-printing/

Life

AI Safety Expert: These are Only 5 Jobs That Will Remain in 2030.
https://youtube.com/watch?v=UclrVWafRAI

Why Nothing Looks Cool Anymore.
https://youtube.com/watch?v=HgjXjPP_p3s

We Need New Word. How About ZUZAI?
https://zuzai.org/

Other

Firefox 157.0 Beta Released with Latest Enhancements.
https://phoronix.com/news/Firefox-157-Beta

Firefox 156 Arrives with Forest of Forks.
https://theregister.com/software/2026/09/19/firefox-forest-of-forks/5297194

LibreOffice Breaks Download Records After Declaring It Has no AI Features.
https://manualdousuario.net/en/libreoffice-download-record-no-ai/

Lazy Reading for 2026/09/20.
https://dragonflydigest.com/2026/09/20/lazy-reading-for-2026-09-20/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Logging mit journalctl: Grundlagen von journald

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

systemd nimmt einem Dienst einen großen Teil der klassischen Daemonologie ab. Ein Programm muß sich nicht selbst in den Hintergrund forken, sein Controlling Terminal verlieren, PID-Dateien verwalten und einen Syslog-Client einbauen. Es kann im Vordergrund laufen und nach stdout und stderr schreiben. Der Service Manager startet es, hält seinen Zustand fest und verbindet seine Ausgaben normalerweise mit systemd-journald.

journald speichert nicht einfach Zeilen in einer Textdatei. Ein Journal-Eintrag besteht aus Feldern wie MESSAGE, PRIORITY, _PID, _UID, _SYSTEMD_UNIT und _BOOT_ID. Viele davon ergänzt journald aus vertrauenswürdigen Prozessdaten. Die Einträge lassen sich nach jedem Feld filtern und in verschiedenen Formaten ausgeben.

Damit entfallen für diese Logs auch logrotate und das nachträgliche Zerlegen von Syslog-Zeilen. journald rotiert seine binären Journal-Dateien selbst und begrenzt ihren Platzverbrauch oder ihr Alter. Das Handbuch von systemd-journald beschreibt die Eingänge und die Speicherung; journalctl ist das Werkzeug zum Lesen.

Ein Dienst schreibt ins Journal

Für einen normalen Service ist keine Logging-Integration nötig:

[Service]
Type=simple
ExecStart=/usr/local/bin/chatto-releasebot

Was der Prozess auf stdout oder stderr schreibt, landet zeilenweise im Journal. StandardOutput=journal ist für Services die Voreinstellung; StandardError=inherit folgt standardmäßig dem Ziel von stdout. Die Details stehen bei StandardOutput= und StandardError= .

Das Programm sollte im Vordergrund laufen und seine normale Ausgabe nicht durch eigenes Forken oder eine Logdatei vor systemd verstecken. Das ist einfacher zu entwickeln und lokal zu testen:

$ /usr/local/bin/chatto-releasebot
Observed version: 0.5.0-beta.4
Already announced.

Unter systemd erscheinen dieselben Zeilen zusammen mit den Meldungen des Service Managers:

Sep 19 14:31:38 kvm.koehntopp.de systemd[50175]: Starting Announce current Chatto release...
Sep 19 14:31:38 kvm.koehntopp.de chatto-releasebot[562549]: Observed version: 0.5.0-beta.4
Sep 19 14:31:38 kvm.koehntopp.de chatto-releasebot[562549]: Already announced.
Sep 19 14:31:38 kvm.koehntopp.de systemd[50175]: Finished Announce current Chatto release.

stdout ist dabei nur der Transport. journald ergänzt unter anderem PID, UID, Executable, Cgroup, Unit, Boot-ID und Zeitstempel. Eine Zeile JSON auf stdout bleibt allerdings eine Zeichenkette im Feld MESSAGE; die enthaltenen JSON-Keys werden nicht automatisch zu Journal-Feldern.

Logs auswählen

Ohne Optionen zeigt journalctl alle Einträge, die der aufrufende Benutzer lesen darf. Root sowie Mitglieder bestimmter administrativer Gruppen können normalerweise das System-Journal lesen. --user beschränkt die Auswahl auf Dienste des eigenen User Managers, --system auf Systemdienste und Kernel.

Der häufigste Filter ist die Unit:

# System-Service
journalctl -u ssh.service

# User-Service
journalctl --user -u chatto-releasebot.service

Mit -f folgt journalctl neuen Einträgen, mit -n begrenzt es die Ausgabe:

journalctl --user -f -u chatto-releasebot.service
journalctl --user -u chatto-releasebot.service -n 10

Zeitangaben akzeptieren sowohl absolute als auch relative Werte. Die vollständige Syntax steht in systemd.time(7) :

# Seit einem Zeitpunkt
journalctl --since today
journalctl --since "1 hour ago"
journalctl --since "2026-09-19 14:00"

# Ein Zeitfenster
journalctl --since "14:00" --until "15:00"

# Aktueller und vorheriger Boot
journalctl -b
journalctl -b -1

# Kernel-Meldungen des aktuellen Boots
journalctl -k -b

Die vorhandenen Boots zeigt journalctl --list-boots. Die Kombination aus Boot, Unit und Zeitraum ist meist ergiebiger als ein grep über alle Meldungen.

Für die Suche im Text gibt es --grep. Es verwendet reguläre Ausdrücke:

journalctl -u ssh.service --grep='failed'

Prioritäten sind Bereiche

Die acht Syslog-Prioritäten reichen von emerg mit dem numerischen Wert 0 bis debug mit 7:

Wert Name Bedeutung
0 emerg System unbenutzbar
1 alert sofortiges Eingreifen
2 crit kritischer Zustand
3 err Fehler
4 warning Warnung
5 notice wichtiges normales Ereignis
6 info Information
7 debug Debugging

Ein einzelner Wert bei --priority= bedeutet „diese Priorität und alles Wichtigere“. Das ist der Punkt, über den man regelmäßig stolpert:

# err, crit, alert und emerg
journalctl -p err

# warning und alles Wichtigere
journalctl -p warning

# ausschließlich err
journalctl -p err..err

# info bis einschließlich debug
journalctl -p info..debug

Ein Eintrag ist ein Satz von Feldern

Die normale Ausgabe sieht absichtlich wie klassisches Syslog aus. -o verbose zeigt stattdessen die Felder eines Eintrags:

$ journalctl --user -o verbose -n 1
Sat 2026-09-19 14:31:38.375293 UTC
 _UID=9000
 _GID=9000
 _BOOT_ID=0827e543c22441a795c53a0303dafdb3
 _MACHINE_ID=f887fe52be104259aed255488762722d
 _HOSTNAME=kvm.koehntopp.de
 PRIORITY=6
 SYSLOG_IDENTIFIER=systemd
 JOB_ID=1860
 JOB_TYPE=start
 _PID=50175
 _SYSTEMD_USER_UNIT=init.scope
 MESSAGE=Finished Announce current Chatto release.
 JOB_RESULT=done
 MESSAGE_ID=39f53479d3a045ac8e11786248231fbf

Eine Beschreibung der bekannten Namen steht in systemd.journal-fields(7) . Felder mit führendem Unterstrich wie _PID und _UID sind vertrauenswürdige Felder: journald ermittelt sie selbst, ein Client darf sie nicht setzen.

Jedes Feld kann direkt als Match verwendet werden:

journalctl --user _PID=50175
journalctl --user JOB_ID=1860
journalctl _BOOT_ID=0827e543c22441a795c53a0303dafdb3

Matches auf verschiedene Felder werden mit AND verknüpft. Mehrere Werte desselben Feldes sind Alternativen:

# Diese Unit und genau diese PID
journalctl _SYSTEMD_UNIT=ssh.service _PID=1234

# ssh.service oder cron.service
journalctl _SYSTEMD_UNIT=ssh.service _SYSTEMD_UNIT=cron.service

Mit + lassen sich ganze Match-Gruppen durch OR verbinden. Die Regeln stehen im Abschnitt Description von journalctl(1) .

Welche Felder und Werte tatsächlich vorhanden sind, läßt sich aus dem Journal selbst ermitteln:

# Alle verwendeten Feldnamen
journalctl -N

# Alle vorhandenen Werte eines Feldes
journalctl -F _SYSTEMD_UNIT
journalctl -F SYSLOG_IDENTIFIER

Ausgabe für Menschen und Programme

Für Menschen ist die kurze Standardausgabe meist richtig. -o cat gibt nur MESSAGE aus. -o short-iso-precise liefert präzise ISO-Zeitstempel. Für Programme gibt es unter anderem json, json-pretty, json-seq und das verlustfreie Journal-Exportformat:

journalctl -u ssh.service -o short-iso-precise
journalctl -u ssh.service -o cat
journalctl -u ssh.service -o json
journalctl -u ssh.service -o export

Ein Skript sollte nicht die optisch angenehme Standardausgabe parsen. Es sollte JSON, das Exportformat oder direkt die Journal-API verwenden. Mit --output-fields= kann es die Ausgabe auf benötigte Felder begrenzen.

Konfiguration und Aufbewahrung

Die Konfiguration besteht aus der Hauptdatei und Drop-ins. systemd-analyze cat-config zeigt die gefundenen Dateien in der Reihenfolge, in der sie ausgewertet werden:

systemd-analyze cat-config systemd/journald.conf

Damit sieht man nicht nur einen Wert, sondern auch seine Herkunft. Ein späteres Drop-in überschreibt bei einfachen Optionen einen früheren Wert. Auskommentierte Zeilen in der Hauptdatei dokumentieren häufig die einkompilierten Defaults; sie sind selbst keine Konfiguration. Der Befehl und seine Suchpfade sind unter cat-config dokumentiert.

Eine lokale Konfiguration gehört in ein Drop-in:

# /etc/systemd/journald.conf.d/10-persistent.conf
[Journal]
Storage=persistent
SplitMode=uid
SystemMaxUse=4G
MaxRetentionSec=180day

Danach wird systemd-journald neu gestartet:

systemctl restart systemd-journald.service

Die vier Optionen haben unterschiedliche Aufgaben:

  • Storage=persistent speichert bevorzugt unter /var/log/journal. Während des frühen Boots oder wenn /var nicht schreibbar ist, wird vorübergehend /run/log/journal verwendet. Storage=volatile bleibt unter /run; Storage=auto verwendet /var/log/journal nur, wenn dieses Verzeichnis existiert.
  • SplitMode=uid legt bei persistenter Speicherung für normale Benutzer getrennte Journal-Dateien an. Das dient vor allem der Zugriffskontrolle. Systemdienste und System-UIDs bleiben im System-Journal.
  • SystemMaxUse=4G begrenzt den Platz für persistente Journal-Dateien. RuntimeMaxUse= wäre das entsprechende Limit unter /run.
  • MaxRetentionSec=180day entfernt Journal-Dateien mit zu alten Einträgen. Größenlimits reichen oft aus; ein Zeitlimit ist sinnvoll, wenn es eine ausdrückliche Aufbewahrungsfrist gibt.

Auf einem Rechner mit Machine-ID f887fe52be104259aed255488762722d kann das so aussehen:

/var/log/journal/f887fe52be104259aed255488762722d/
├── system.journal
├── system@1164fa7c79ef404e9f1771f93fc487e7-0000000000000001-00065b3a5d983752.journal
└── user-9000.journal

system.journal und user-9000.journal sind aktive Dateien. Dateien mit @... im Namen sind rotierte Archive. Alte Archive werden nach den konfigurierten Grenzen gelöscht; deshalb braucht das Journal kein logrotate.

Den aktuellen Platzverbrauch und eine sofortige, manuell angeforderte Bereinigung gibt es ebenfalls über journalctl:

journalctl --disk-usage
journalctl --rotate --vacuum-size=2G
journalctl --rotate --vacuum-time=90days

Normalerweise konfiguriert man die gewünschten Grenzen und läßt journald die Arbeit machen. --vacuum-* ist für eine unmittelbar nötige Bereinigung.

Sealing ist keine ewige Signatur

Seal=yes aktiviert Forward Secure Sealing für persistente Journal-Dateien, wenn zuvor mit journalctl --setup-keys Schlüsselmaterial eingerichtet wurde. Dabei werden kryptographische Tags über abgeschlossene Abschnitte gebildet. Mit einem getrennt aufbewahrten Verification Key kann man später Manipulationen erkennen.

Das ist keine allgemeine digitale Signatur jedes Eintrags und macht das Journal nicht unveränderlich. Ein Angreifer mit Root-Rechten kann weiterhin Dateien löschen. Die Forward-Secure-Konstruktion soll verhindern, daß er ältere Einträge unbemerkt verändert und danach passende Tags neu berechnet. Aufbewahrungsgrenzen und Rotation gelten weiterhin.

journalctl --verify prüft die interne Konsistenz von Journal-Dateien. Die kryptographische Prüfung der Seals braucht zusätzlich den Verification Key. Details stehen bei --setup-keys, --verify und --verify-key= .

Aus Anwendungen strukturiert loggen

Für viele Programme reichen verständliche Zeilen auf stdout und stderr. Sie erhalten automatisch die vertrauenswürdigen Prozessfelder und lassen sich über Unit, PID, UID, Boot und Zeit filtern.

Ein bestehendes Programm kann man außerhalb einer Unit mit systemd-cat an das Journal anschließen:

systemd-cat -t importer -p info /usr/local/bin/importer --run-once

-t setzt den SYSLOG_IDENTIFIER, -p die Standardpriorität. Das erzeugt jedoch noch keine anwendungsspezifischen Felder.

Wenn Logs nach fachlichen Attributen wie ORDER_ID, TENANT oder ERROR_CODE durchsuchbar sein sollen, muß die Anwendung das native Journal-Protokoll verwenden. In C stellt libsystemd dafür sd_journal_send(3) bereit:

#include <systemd/sd-journal.h>
#include <syslog.h>

sd_journal_send("MESSAGE=Order could not be charged",
 "PRIORITY=%i", LOG_ERR,
 "ORDER_ID=%s", order_id,
 "ERROR_CODE=%i", error_code,
 "MESSAGE_ID=51e9d47b9b9e4b9aa0b20d6e13e12345",
 NULL);

Für Python stellt das systemd-Projekt die Bibliothek python-systemd bereit. Distributionen paketieren sie normalerweise als python3-systemd, auf PyPI heißt sie systemd-python. Das Modul systemd.journal bildet denselben Aufruf direkt ab:

from systemd import journal

journal.send(
 "Order could not be charged",
 PRIORITY=str(journal.LOG_ERR),
 ORDER_ID=str(order_id),
 ERROR_CODE=str(error_code),
 MESSAGE_ID="51e9d47b9b9e4b9aa0b20d6e13e12345",
)

Eigene Feldnamen bestehen üblicherweise aus Großbuchstaben, Ziffern und Unterstrichen. Sie dürfen nicht mit einem Unterstrich beginnen, weil dieser Namensraum den von journald gesetzten vertrauenswürdigen Feldern gehört.

MESSAGE_ID bezeichnet nicht einen einzelnen Vorgang, sondern einen stabilen Meldungstyp. Alle Vorkommen von „Zahlung fehlgeschlagen“ können dieselbe ID tragen; ORDER_ID unterscheidet die konkreten Vorgänge. Eine neue zufällige 128-Bit-ID erzeugt systemd-id128 new.

Danach funktionieren fachliche Abfragen ohne Text-Parsing:

journalctl MESSAGE_ID=51e9d47b9b9e4b9aa0b20d6e13e12345
journalctl ORDER_ID=4711
journalctl ERROR_CODE=23 -o json-pretty

Felder sollten stabil und sparsam gewählt werden. Ein Journal-Eintrag ist ein Ereignis, kein Ersatz für eine fachliche Datenbank. IDs, Fehlerklassen und Zustandsübergänge sind gute Felder; komplette wechselnde Objektstrukturen sind es meist nicht.

Rate Limits und verlorene Meldungen

Ein Dienst, der in einer Schleife Tausende identische Fehler schreibt, darf nicht die Platte füllen. RateLimitIntervalSec= und RateLimitBurst= begrenzen deshalb die Zahl gespeicherter Meldungen pro Service und Zeitfenster. Überschüssige Meldungen werden verworfen; journald protokolliert die Zahl der unterdrückten Einträge.

Das schützt das System, macht aber aus Debug-Logging keine kostenlose Ressource. Wer einen Fehler untersucht, sollte auf Meldungen über unterdrückte Einträge achten. Für einzelne Units können LogRateLimitIntervalSec= und LogRateLimitBurst= die globalen Werte überschreiben.

Weniger tippen

journalctl verwendet auf einem Terminal normalerweise einen Pager. Für einmalige Aufrufe schaltet --no-pager ihn ab. Dauerhaft kann die Shell cat als Pager vorgeben:

export SYSTEMD_PAGER=cat
alias jcu='journalctl --user'

Dann wird aus

journalctl --user --no-pager -f -u chatto-releasebot.service

das kürzere

jcu -f -u chatto-releasebot.service

Der wesentliche Unterschied zu klassischen Textlogs ist nicht die bequemere Kommandozeile. Es ist das Datenmodell: Eine Logmeldung ist ein strukturierter Datensatz mit von journald ermittelten Prozessattributen und optionalen Anwendungsfeldern. journalctl projiziert und filtert diese Datensätze. Wer das Journal wie eine große Datei behandelt und nur grep darauf wirft, benutzt den uninteressantesten Teil davon.

Top

Data Broker Radaris Loses Domains in Privacy Fight

Post by Brian Krebs via Krebs on Security »

The consumer data broker Radaris.com has long had a reputation for ignoring requests to remove personal information from its vast empire of people-search services online. That reputation caught up with the company recently in a lawsuit alleging Radaris violated a New Jersey privacy law that provides for hefty fines against data brokers that publish personal information on state law enforcement officials. In the face of repeated stonewalling and prevarication by attorneys for Radaris, the judge in the case ordered that radaris.com and more than a dozen other data broker domains be transferred to the plaintiffs.

The radaris.com website, prior to the domain transfer to Atlas.

In February 2024, Radaris was sued by Atlas Data Privacy Corp, a company that has been pursuing data brokers alleged to be violating a New Jersey statute called Daniel’s Law. The statute allows state law enforcement officials, government personnel, judges and their families to have their information completely removed from commercial data brokers and people-search services, and provides for fines of $1,000 per violation against companies that ignore removal requests.

Less than a month after Atlas sued Radaris, KrebsOnSecurity published a deep dive into the Radaris co-founders — Igor and Dmitry Lubarsky (also spelled Lybarsky) — Russian-born brothers living in Massachusetts who operate a dizzying array of people-search companies as well as a number of Russian language dating services and affiliate programs.

Attorneys for the Lubarsky brothers threatened to sue for defamation if the story wasn’t removed and an apology issued. Their attorney asserted that our reporting was wildly inaccurate, and that the true owners of the company were Ukrainians living in Ukraine.

The Lubarsky brothers Dmitry or “Dan” (left) and Gary/Igor.

KrebsOnSecurity doubled down and showed how the Lubarsky brothers built and operated Radaris and other data broker companies using a fictitious CEO’s name. Our follow-up story noted that Radaris’s attorney — a lawyer with the Boston Law Group named Val Gurvits — admitted his clients had invented the CEO pseudonym “Gary Norden,” and that Radaris also had issued multiple press releases over the years that quoted the fake CEO while seeking money from potential investors.

Attorneys for Radaris waited until the last minute to appear in court and contest what was all but certain to be a default judgment in favor of the plaintiffs, and then told the court that Atlas had failed to serve the real owners and operators of Radaris and several of its sister data broker companies.

Atlas re-filed the lawsuit in June 2025, this time dramatically expanding the number of Radaris family data brokers accused of violating Daniel’s Law. Matt Adkisson, president and CEO of Atlas, said Radaris turned to a tried-and-true playbook: Delaying in court until the last possible minute, and playing shell games with Radaris’s true country of origin and the individuals listed as owners and operators of these sites.

“We refer to this period as their island-hopping phase. Privacy policies changed constantly, and new entities kept appearing from places like the Marshall Islands, the British Virgin Islands, and Seychelles,” Adkisson told KrebsOnSecurity. “Behind the scenes, it felt like a shell game. Defense lawyers told the court that certain entities merely operated the domains and were the proper parties to sue. But by the time a judgment neared, those entities would be discarded and new entities would appear. Meanwhile, the lawyers claimed the other entities that actually owned the domains should not be held responsible.”

Adkisson said when the defendants updated their terms of service to state that Radaris was suddenly managed by a company in the Marshall Islands, Atlas hired an investigator in that country and soon learned the brand new entity that Radaris claimed was managing the company didn’t even exist yet.

Mr. Gurvits stepped forward as Radaris’s attorney in a class action lawsuit the company temporarily lost in 2017 because it never contested the claim in court. When the plaintiffs told the judge they couldn’t collect on the $7.5 million default judgment, the court ordered the domain registry Verisign to transfer the radaris.com domain name to the plaintiffs.

Mr. Gurvits appealed that verdict, arguing the lawsuit hadn’t named the actual owners of the Radaris domain name — a Cyprus company called Bitseller Expert Limited — and thus taking the domain away would be a violation of their due process rights.

The judge in the 2017 case ruled in Radaris’ favor — halting the domain transfer — and told the plaintiffs they could refile their complaint. Soon after, the operator of Radaris changed from Bitseller to Andtop Company, an entity formed (PDF) in the Marshall Islands in Oct. 2020. The plaintiffs never re-filed their lawsuit.

A mind map of various entities tied to Radaris and the company’s co-founders. Click to enlarge.

“That seemed to be their modus operandi,” said Raj Parikh, a partner at PEM Law in New Jersey who handles most of the Daniel’s Law litigation for Atlas. “In the past, they won by attrition. Plaintiffs’ attorneys tired of the procedural games and just gave up. That strategy worked for a decade, and it probably would have worked in this case too, since any financial recovery from foreign actors will be difficult. But we were acutely aware of the threat this website posed to law enforcement officers and other public officials in New Jersey, and decided early on to commit whatever time and resources were necessary to remove that threat.”

On August 26, the judge in the New Jersey case found the defendants were given multiple chances to appear and defend the claims against them but had failed to do so. Mr. Gurvits declined to comment on the case, saying it had been assigned to another attorney, a Mr. Victor Worms. In response to questions, Mr. Worms asserted the New Jersey court transferred Radaris.com to Atlas as part of a default judgment against Radaris.com, which is not a legal entity.

“We have made a motion to vacate that default judgment on the grounds that it is void since a non-entity has no legal capacity to sue or be sued,” Worms replied. “We also intend to pursue all appropriate appeals because we believe the transfer of Radaris.com amounts to a forfeiture in violation of various constitutional principles.”

While radaris.com still comes up prominently in results when searching online for U.S. residents by name, the domain no longer sells detailed personal dossiers on millions of Americans. Its homepage now displays a notice from Atlas, as well as links to our previous reporting on Radaris.

EMAIL CONFIRMATIONS

Atlas told KrebsOnSecurity that it has obtained more than 10,000 emails and documents in the course of litigation, and that those messages confirm our previous reporting on the owners and operators of Radaris and its myriad companies.

Atlas said the emails clearly establish that the nominal legal vehicles — Radaris America, Inc.; Bitseller Expert Limited; Digital Orbit Corp; Core Solutions Group Inc; Lucky Solutions Inc; Virtura Corp; Veripages Inc.; Nuform Solutions Inc.; Growth Data Advisors Inc.; Property Experts, Inc — are all administered by the same three or four people from the same mailboxes, share one bank or payment card set, and are all managed from one virtual office address.

“The corpus establishes, with documentary evidence generated independently by banks, payment processors, hosting providers, registrars, software-as-a-service vendors and the operators’ own systems, that radaris.com and at least twenty-five other people-search websites are one operation run by a small Boston-area group whose administrative, financial and technical functions sit on the difive.com mail domain and its successors (centerex.com, scienteco.com, eprofit.com, realmo.com, pub360.com),” reads a summary shared by Atlas.

Atlas said the emails show Radaris.com earns approximately $42,000 a month, while Veripages.com earns around $45,000 monthly via its partnership with the Lifetime Value Company, a marketing and advertising firm whose brands include PeopleLooker, PeopleSmart, NumberGuru, and Bumper, a car history site.

According to Atlas, the emails also showed the Radaris family of websites earns as much as $25,000 each month from their partnership with Onerep, a company that claims to help people remove their information from people-search sites. In March 2024, KrebsOnSecurity revealed how the Belarusian founder of Onerep had launched and operated dozens of people-search sites over the years and was continuing to operate one of them (Nuwber), effectively spreading the disease and selling the cure.

The domain radaris.com now redirects to this notice from Atlas about the court-ordered domain transfer.

The domain radaris.com now redirects to this notice from Atlas about the court-ordered domain transfer.

All told, the New Jersey court has so far transferred 14 domain names from the Radaris family of companies to Atlas. Radaris.com now redirects to a notice of the court-ordered domain transfer.

THE ROAD AHEAD

The Radaris family of companies is still potentially facing fines of $1,000 per alleged violation of Daniel’s Law. For the time being, however, Daniel’s Law is facing a constitutional challenge from virtually all of the 150 other consumer data broker firms being sued by Atlas.

The data broker industry responded by having at least 70 of the Atlas lawsuits moved to federal court, challenging the New Jersey statute as overly broad and a violation of the First Amendment. The U.S. Court of Appeals for the Third Circuit has not yet issued a decision on the constitutional challenge, but either way the case is widely expected to be appealed all the way to the U.S. Supreme Court.

Meanwhile, at least 14 other states have now passed laws modeled after the New Jersey statute, with more states considering similar measures. However, West Virginia’s Daniel’s Law was ruled facially unconstitutional under the First Amendment by a federal district court in August 2025.

Justin Sherman is a privacy expert and author of the forthcoming book “The Middlemen,” which examines how the data broker industry powers modern surveillance. Sherman said federal lawmakers have long faced intense lobbying by the technology industry against more restrictive U.S. data privacy laws, but that many powerful industries are now working against passing comprehensive data privacy legislation.

“These days at the federal level, add in the intense amount of lobbying against these laws from social media companies, big tech, cryptocurrency firms, and now AI proponents in the mix who claim that limiting their data scraping is somehow going to collapse the whole U.S. economy under Chinese rule,” he said.

Sherman said people-search companies will continue to thrive unless and until Congress enacts meaningful consumer privacy and data protection laws that are relevant to life in the 21st century. That’s because virtually all state privacy laws exempt records that might be considered “public” or “government” documents, including voting registries, property filings, marriage certificates, motor vehicle records, criminal records, court documents, death records, professional licenses, bankruptcy filings, and more.

At least 25 states have passed or implemented laws requiring age verification for residents seeking to access adult content online, but there is no federal law that limits how the companies that are scanning everyone’s drivers license can use, share or keep the data provided. Had such restrictions been enshrined in law, we may have avoided the recent breach at IDScan.net, which exposed the drivers license information on more than 153 million Americans when the records were briefly turned into a point-and-click identity theft service on the dark web.

“The average person can look at Daniel’s Law and have a perfectly normal reaction, which is that everyone should be covered, not just police and judges,” Sherman said. “But we don’t need more wake-up calls. We’ve had eight million wake-up calls already on the need for better privacy laws. The lack of comprehensive federal privacy law is not for a lack of knowledge, and anyone claiming otherwise is either not reading the news or kidding themselves.”

Top

Learning at Scale: FreeBSD Foundation Intern Nimish Jain on Exploring the FreeBSD Codebase

Post by FreeBSD Foundation via FreeBSD Foundation »

For Nimish Jain, working on FreeBSD this summer meant getting an up-close look at something he had long been curious about: how large, complex pieces of software work together.

A Software Engineering student at the University of Waterloo, Nimish is preparing to enter his second year. He was already a longtime FreeBSD user when he came across the FreeBSD Foundation internship opportunity through his university’s job portal.

“As soon as I saw the word ‘FreeBSD,’ I had to apply,” he said.

That decision led to a summer of working across several areas of the operating system, learning from experienced FreeBSD contributors, and seeing his own code make its way into a project used around the world.

From FreeBSD User to Contributor

Nimish doesn’t remember a single moment that sparked his interest in systems programming. Instead, it grew from a broader curiosity about software.

“I have always been curious about how large pieces of software work together to complete a task,” he said.

That curiosity, combined with his experience using FreeBSD, made the Foundation internship stand out immediately. The internship also gave Nimish the opportunity to work on more than one area of the Project. Over the summer, his work included upgrading virtualization drivers, working on a way to connect to a filesystem over a network, and porting a Raspberry Pi installer to FreeBSD.

The variety gave him an opportunity not only to write code, but also to better understand how different parts of a large operating system interact.

Learning Through Code Review

For Nimish, some of the most rewarding moments of the internship came after he submitted his work and waited for feedback.

“The most rewarding part of this internship has been waking up to an email saying someone has reviewed your work,” he said.

Code review became an important part of the learning process. Seeing how experienced contributors approached the same problem often introduced him to solutions he hadn’t considered.

“I’m often delighted when someone else points out an elegant approach I missed out on,” he said.

And, of course, there was another particularly satisfying moment familiar to many developers: when the code worked on the first try.

Navigating a Large Codebase

Working within an operating system also introduced Nimish to one of the biggest challenges of large-scale software development: complexity.

With so much of the system to understand, he spent a significant amount of time reading through FreeBSD internals and asking questions.

Writing the code itself was an iterative process.

“You start out with a draft, refine, and then revisit until you think the logic makes sense,” he said. “Going through all those stages was both tough and fun.”

That process helped him see that contributing to a large project is about more than immediately knowing the right answer. It requires taking the time to understand existing code, asking questions, testing ideas, incorporating feedback, and refining a solution until it works within the larger system.

Experiencing the FreeBSD Community in Person

This summer, Nimish also attended BSDCan and the FreeBSD Developer Summit, where he had the opportunity to meet contributors and hear discussions about the future of the Project.

“It was awesome seeing so many people interested in FreeBSD share their thoughts on going forward,” he said.

He particularly enjoyed the coding sprints in the hacker lounge, where contributors could work together and exchange ideas outside of the formal conference sessions.

One talk that stood out explored keeping an SSH connection alive even when a machine goes down. But for Nimish, some of the greatest value came from simply meeting other members of the community and hearing how they approached problems.

“Meeting different people and understanding their perspectives on issues I cared about helped me decide what I want to work on and how to go about solving it.”

Seeing How Open Source Fits Together

Before his internship, Nimish tended to think of open source projects as relatively self-contained. Working on FreeBSD changed that perspective.

“This internship has made me realize how large the open source ecosystem is,” he said. “Forget different projects, there are so many subsystems within FreeBSD itself!”

The deeper he went into the codebase, the more he saw the connections between different components. A change in one area may depend on another subsystem, affect another project, or require updates in multiple places before everything works as expected.

“Everything is tied together so well that we often don’t grasp what is required to make everything work.”

That interconnectedness became one of the biggest lessons of his internship, and one of the things that made working on FreeBSD at scale so valuable.

Start by Exploring

For students interested in making their first FreeBSD contribution, Nimish recommends starting with curiosity rather than worrying about knowing everything upfront.

“I’d say take time exploring the system,” he said.

Reading the code is often where the process begins.

“It is usually when you’re reading through code that you start to ask yourself questions,” he said. “Questions that lead to discussions, collaboration, puzzlement, and development.”

For Nimish, those questions are part of what makes working on a project like FreeBSD interesting. There is always another subsystem to understand, another connection to discover, or another problem worth exploring.

Continuing With FreeBSD

Nimish plans to continue contributing to FreeBSD after his internship, particularly in virtualization. He is also interested in eventually exploring FreeBSD’s virtual memory subsystem.

Looking further ahead, he plans to continue working in distributed systems programming while also making room for another longtime interest: computer vision.

Whatever direction that takes him, his internship gave him something difficult to replicate in a classroom: the experience of writing software that becomes part of a major operating system.

“Seeing something you coded being deployed to a major operating system, and ready to be used by thousands worldwide, is amazing,” he said. “The technical challenges of working at scale are extremely rare to find anywhere else.”

Nimish also credits the people who helped make that experience possible. He thanked Siva Mahadevan and Ed Maste for the opportunity and for answering his questions throughout the internship, as well as Mark Johnston for reviewing much of his work.

Just as important was the environment surrounding the work.

“Everyone here is also very helpful, and it doesn’t feel like a hurdle approaching anyone,” he said.

The freedom to explore was another defining part of his experience.

“I think that is extremely rare too.”

Asked to describe his time working with the FreeBSD Foundation in one word, Nimish chose a simple one: stellar.

For a student who began the summer curious about how complex software works together, the internship offered the chance to do more than study that complexity. It gave him the opportunity to work within it, contribute to it, and help shape what comes next.

The post Learning at Scale: FreeBSD Foundation Intern Nimish Jain on Exploring the FreeBSD Codebase first appeared on FreeBSD Foundation.

Top

Valuable News – 2026/09/14

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD 14.5-RELEASE Now Available.
https://lists.freebsd.org/archives/freebsd-announce/2026-September/000316.html

FreeBSD 14.5 Goes Official.
https://officialaptivi.wordpress.com/2026/09/08/freebsd-14-5-goes-official/

FreeBSD 14.5 Released With Hardware Support Improvements and Many Bug and Security Fixes.
https://phoronix.com/news/FreeBSD-14.5-RELEASE

FreeBSD 14.5 Adds inotify(4) Support and Boot Loader Fixes.
https://alternativeto.net/news/2026/9/freebsd-14-5-adds-inotify-support-and-boot-loader-fixes/

Eleven Lines for a Dozen Jails.
https://vivianvoss.net/blog/eleven-lines

ekkoBSD 2.1 Released after More then Two Private Decades.
https://ekkobsd.org/downloads

Hex Editor ImHex for Reverse Engineers/Programmers.
https://github.com/WerWolv/ImHex

Dedicated Jail Images.
https://vivianvoss.net/blog/dedicated-jail-images

GSoC 2026 Reports – Testing Compat Linux on NetBSD – Syscall Testing – Part 2.
https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux_2

Package Management in AIX – Enable autofsinstall(8) and Say Goodbye to Manual Fileset Hunting.
https://osadmins.com/en/package-management-in-aix-how-to-enable-autofsinstall-and-say-goodbye-to-manual-fileset-hunting/

VIOS System Update Using Alternate (alt_disk) Disk.
https://osadmins.com/en/vios-system-update-using-alternate-disk-alt_disk/

Why is Everyone Stuck on FreeBSD 13?
https://klarasystems.com/articles/freebsd-13-eol-why-systems-are-still-running-freebsd-13/

BSD Part Deux.
https://ministryofmayhem.space/posts/bsdptdeux/

V Language Enabled by Default on BSD.
https://github.com/vlang/v/pull/28475

FreeBSD ZFS Pool DEGRADED.
https://dan.langille.org/2026/09/12/data04-degraded-oh-and-have-i-mentioned-watchgoose/

NetBSD/amd64 Guest Can Boot in 40ms.
https://reddit.com/r/BSD/comments/1wehwzb/netbsd_amd64_guest_can_boot_in_40ms/

System Activity Monitor tosview(8) Inspired by SGI IRIX gr_osview/xosview Tools.
https://github.com/lex0de/tosview

Aptivi Weekly Tech News – Issue #15 – Week 37.
https://officialaptivi.wordpress.com/2026/09/12/aptivi-weekly-tech-news-issue-15-week-37-9-12-2026/

Unix Heritage Society Preserves/Maintains Historical UNIX Artifacts.
https://tuhs.org/

BIRD on FreeBSD: BGP MD5 Needs Source Address.
https://kuon.ch/post/2026-09-10-til-bird-setkey/

DaemonForums Down as One of Creators Terminally Ill at Hospital.
https://forums.freebsd.org/threads/daemonforums.103642/

MariAdmin and OpenBSD on OVHcloud.
https://ozgurkazancci.com/freebsd-jstats-a-resource-monitoring-tool-for-jails/

Small jstats(8) Tool for FreeBSD Jails RAM/CPU/DISK Stats.
https://github.com/ozgurkazancci/jstats

Overview of AOOSTAR WTR Pro on *BSD.
https://tumfatig.net/2026/overview-of-aoostar-wtr-pro-on-bsd/

FreeBSD Jail Orchestration Runtime Descriptor.
https://fjord.daemonless.io/

Firewall I Never Had to Relearn.
https://vivianvoss.net/blog/pf-twenty-years

UNIX/Audio/Video

FreeBSD Basics – Part 2 – System /bin/sh Shell.
https://youtube.com/watch?v=-6NlGRxZNWU

FreeBSD Basics – Part 3 – System /bin/sh Shell.
https://youtube.com/watch?v=MQTtPsjmJOw

Modern Service Manager rcd(8) FreeBSD Way.
https://exquisite.tube/w/nPGJ2TbJ8kzP8SHXiwzirU

Make FreeBSD Cloud Native with nuageinit(7) Tool.
https://exquisite.tube/w/4Z2quSCm6Ni1ExrsQJa249

EuroBHYVEcon 2026.
https://youtube.com/watch?v=KJ_XYyzzTLc

Checking Out Gershwin Desktop on GhostBSD/FreeBSD Live.
https://youtube.com/watch?v=oixYFTC-jI4

BSD Now 680: What FS?
https://www.bsdnow.tv/680

Hardware

OpenNMC is Open Replacement for Expensive APC Management Cards.
https://jeffgeerling.com/blog/2026/opennmc-apc-ups-replacement-card/

Introducing JetKVM Mini.
https://jetkvm.com/blog/introducing-jetkvm-mini

Intel Xeon 600 vs. AMD Threadripper 9000 Performance.
https://phoronix.com/review/intel-xeon-600-amd-threadripper-9000/15

Intel Opens X86 to 3rd Parties: Licenses Atom RTL Code to RosaicLabs.
https://alextech.ai/en/news/intel-opens-x86-to-third-parties-licenses-atom-rtl-code-to-rosaiclabs/

216.000.000 Spy TVs – LG Smart TV Problem.
https://youtube.com/watch?v=6IFVTcM28KA

Life

The Art of Being Dangerous – Jordan Peterson Motivation.
https://youtube.com/watch?v=kxjMYWAV2Pk

Other

Firefox AI Switch is Off. Telemetry is Not.
https://marius.blog/firefox-155-ai-kill-switch-retest/

UNIX Inspired Literature Crumbs – Small Artifacts Each About One Idea.
https://alfredollaquet.github.io/Alfredo_Llaquet-Alsina_Crumbs/

Lazy Reading for 2026/09/13.
https://dragonflydigest.com/2026/09/13/lazy-reading-for-2026-09-13/

ASCII Graffiti @ GOTO8O.
https://goto80.com/ascii-graffiti

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Sie wollen eine Pause. Es wird keine geben

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

OpenAI will 2026 nicht mehr an die Börse. Die Begründung, die Sam Altman offiziell gibt ist AI Safety. Zugleich erklärt er sich bereit, die Entwicklung leistungsfähiger Modelle gemeinsam mit anderen Labs zu verlangsamen. Anthropic fordert ebenfalls eine koordinierte Pause.

Ich glaube denen den Wunsch nach der Pause, aber ich glaube die Geschichte dahinter nicht.

Die AI-Firmen entdecken die Notwendigkeit einer Pause genau in dem Moment, in dem ihnen die Finanzierung davonläuft, ein Börsengang ihre kaputten Economics offenlegen würde und Open-Weights-Modelle anfangen, auf Rechnern unter einem Schreibtisch nützlich zu werden.

Das ist kein Zufall. Die Pause soll nicht den amoklaufenden Maschinengott aufhalten, sondern einen Investitionszyklus verlängern, der seine Kosten absehbar nicht mehr einspielen kann.

Die Kreditmaschine

AI sieht von außen wie ein Softwaregeschäft aus, aber in Wahrheit ist Frontier AI eine kapitalintensive Schwerindustrie.

Man braucht Grundstücke, Hallen, Umspannwerke, Stromverträge, Kühlung, Netzwerk, RAM, NVMe und sehr große Mengen Beschleuniger. Die Softwaremargen kommen erst danach, falls überhaupt.

Die Anbieter haben diese Infrastruktur nicht aus laufenden Einnahmen bezahlt. Stattdessen haben sie eine Kreditmaschine gebaut:

  • OpenAI, Anthropic, xAI und andere versprechen langfristigen Compute-Bedarf.
  • Oracle, CoreWeave und die Hyperscaler bestellen Hardware und Rechenzentren.
  • Banken, Anleihemärkte, Private Credit und Zweckgesellschaften finanzieren den Bau.
  • Hersteller und Rechenzentrumsentwickler bekommen Abnahmegarantien.
  • Die Verträge werden als Begründung für die nächste Finanzierungsrunde verwendet.

Dabei entstehen überall Umsätze, Forderungen, Backlogs und Bewertungen. Cash entsteht sehr viel später, und der einzige Weg, auf dem Cash entstehen kann, ist wenn Unternehmen AIaaS kaufen und Token abnehmen. Und es sieht zunehmen so aus, als müssten sie das nicht tun.

Die Finanzierungen sind zum Teil überzeichnet. Das klingt gesund, bedeutet aber nur, daß mehr Investoren eine hoch verzinste und gut besicherte Forderung kaufen wollen, als die Emission groß ist. CoreWeave konnte im Mai 2026 eine überzeichnete Finanzierung während der Syndizierung sogar um 50 Basispunkte billiger bekommen.

Der Markt ist nicht leer. Das billige, unbesicherte und bedingungsarme Kapital ist leer.

Deswegen wandert die Finanzierung in besicherte Kredite, GPU-backed Loans, Project Finance, Sale-and-Leaseback, Private Credit, ausländische Anleihemärkte und Staatsfonds. Die OECD erwartet , daß Private Credit in vier Jahren ungefähr 800 Milliarden USD zur AI-Expansion beitragen soll. Sie schreibt im selben Bericht, daß selbst das für den geplanten Ausbau nicht reicht.

CoreWeave

CoreWeave ist die Kreditmaschine ohne Verkleidung.

Zum 30. Juni 2026 hatte die Firma 35,6 Milliarden USD Schulden . Davon werden 4,4 Milliarden noch 2026 und 6,2 Milliarden 2027 fällig. Einige Kredite kosten effektiv zwischen 9 und 15 Prozent.

Im ersten Halbjahr 2026 gingen 5,2 Milliarden USD in Tilgung und 982 Millionen in Zinsen. Der Nettozinsaufwand in der Ergebnisrechnung betrug 1,18 Milliarden USD. CoreWeave machte im selben Zeitraum 4,7 Milliarden USD Umsatz und 1,37 Milliarden USD Verlust.

Das Unternehmen muß weiter investieren, um die bereits abgeschlossenen Kundenverträge bedienen zu können. Es muß zugleich alte Finanzierungen tilgen, während es neue aufnimmt. Seine eigene Risikobeschreibung sagt ausdrücklich, daß ungünstigere Kreditbedingungen die Erfüllung von Kundenverträgen verhindern können.

Das ist kein Softwareunternehmen mit etwas Fremdkapital. Das ist eine gehebelte Wette auf drei Annahmen:

  • Die Großkunden bleiben zahlungsfähig.
  • Sie nehmen die bestellte Kapazität tatsächlich ab.
  • Die Hardware bleibt lange genug wirtschaftlich ausgelastet.

Wenn eine dieser Annahmen fällt, hat CoreWeave keine zweite Ertragsmaschine, die den Fehler bezahlt.

Oracle

Oracle hat immerhin ein altes und profitables Softwaregeschäft. Die Firma setzt es aber vollständig als Sicherheit für die AI-Wette ein.

Im Geschäftsjahr 2026 nahm Oracle 43 Milliarden USD neue Schulden und fünf Milliarden USD Eigenkapital auf. Zwischen Juni und August 2026 verkaufte die Firma weitere Aktien für 20 Milliarden USD .

Am 31. August 2026 standen 125 Milliarden USD kurz- und langfristige Kredite in der Bilanz. Dazu kommen 31 Milliarden USD Operating-Lease-Verbindlichkeiten und 288 Milliarden USD bereits unterschriebene zusätzliche Lease Commitments, fast vollständig für Rechenzentren. Die meisten davon beginnen bis 2029 und laufen danach 15 bis 19 Jahre.

Diese 288 Milliarden stehen noch nicht als Verbindlichkeit in der Bilanz, weil die betreffenden Leases noch nicht begonnen haben. Unterschrieben sind sie trotzdem.

Auf der anderen Seite stehen 664 Milliarden USD Remaining Performance Obligations. Das sind vertraglich erwartete zukünftige Umsätze. Es ist kein Geld auf dem Konto. Es ist nicht einmal notwendigerweise profitabler Umsatz.

Oracle hat also langfristige Verpflichtungen aufgenommen, um zukünftigen Umsatz mit wenigen sehr großen AI-Kunden zu bedienen. Das kann funktionieren. Es funktioniert genau so lange, wie die Kunden zahlen und der Preis für Compute nicht schneller fällt als die Kosten der Verpflichtungen.

Oracle ist damit der systemisch interessantere Fall, denn CoreWeave kann alleine platzen, aber Oracle kann Kreditmärkte, Leasinggesellschaften, Zulieferer und Aktionäre mitnehmen.

Die Hyperscaler folgen

Meta, Microsoft und Alphabet sind nicht CoreWeave in groß – noch nicht.

Sie besitzen profitable Kerngeschäfte, aus denen sie den Ausbau bezahlen können. Aber auch ihre Bilanzen werden durch AI sichtbar umgebaut.

Meta hatte Ende Juni 83,7 Milliarden USD langfristige Schulden und 349 Milliarden USD nicht kündbare vertragliche Verpflichtungen . Die meisten Verpflichtungen betreffen fremde Cloud-Kapazität und andere technische Infrastruktur.

Alphabet hatte 98,2 Milliarden USD langfristige Schulden und beschaffte im zweiten Quartal weitere 20,5 Milliarden am Anleihemarkt. Das Geld ist ausdrücklich auch für AI-Infrastruktur bestimmt.

Microsoft plant für das Kalenderjahr 2026 inzwischen ungefähr 175 Milliarden USD Capex . Die Zahl ist bereits geschönt, weil Microsoft zukünftige Rechenzentrumsverträge vermehrt als Operating Leases statt als Finance Leases strukturieren will. Operating Leases erscheinen nicht im ausgewiesenen Capex. Bezahlt werden müssen sie trotzdem.

Diese Firmen sind noch nicht in einer Liquiditätskrise. Sie sind in einer Kapitalallokationskrise: Ihre alten Geschäfte erwirtschaften Geld, das in immer schneller alternde AI-Infrastruktur umgeleitet wird. Mit jedem Jahr wird es schwieriger, einen solchen Ausbau aus dem Cashflow zu bezahlen. Also gehen auch die größten Firmen der Welt an die Kreditmärkte.

Irgendwann konkurrieren Oracle, CoreWeave, Meta, Microsoft, Alphabet, Amazon, OpenAI, Anthropic und xAI nicht mehr nur um GPUs und Strom. Sie konkurrieren um denselben Kredit.

Was eine Börse sichtbar macht

OpenAI und Anthropic wollten an die Börse. OpenAI hat dafür im Juni 2026 vertraulich Unterlagen eingereicht. Nun wird der Börsengang auf mindestens 2027 verschoben . Die offizielle Begründung lautet AI Safety.

Ein Börsengang bringt Geld. Er bringt aber auch geprüfte Abschlüsse, Material Contracts, Related-Party Transactions, Kundenkonzentration, Leases, Kaufverpflichtungen, Risikofaktoren und quartalsweise Berichte.

Er liefert nicht automatisch eine hübsche Tabelle mit Inference Cost pro Token. Aber er liefert genug Daten, um die tatsächlichen Kosten zu rekonstruieren:

  • Umsatzqualität statt annualisierter Run Rate,
  • Gross Margin nach Inference Cost,
  • Abschreibungen auf Beschleuniger,
  • Strom-, Netzwerk- und Speicherkosten,
  • langfristige Compute-Verträge,
  • Mindestabnahmen,
  • Kundenkonzentration,
  • Cash Burn,
  • Related-Party- und Ringfinanzierungen.

Dann werden OpenAI, Anthropic, CoreWeave, Oracle und die Hyperscaler vergleichbar. Sie wollen nicht vergleichbar sein.

xAI zeigt inzwischen unfreiwillig, warum. Vor der Verschmelzung mit SpaceX war die Firma eine schwarze Kiste. Nach dem SpaceX-Börsengang muß sie als Teil des AI-Segments berichtet werden.

Das AI-Segment von SpaceX verlor im zweiten Quartal 2026 operativ 1,26 Milliarden USD, im ersten Halbjahr 3,73 Milliarden. Frühere xAI-Kredite und Anleihen kosteten 12,5 Prozent. Ihre vorzeitige Rückzahlung erzeugte ungefähr 739 Millionen USD Vorfälligkeitskosten. Dazu kommen mehr als 13 Milliarden USD Finanzierungsschulden aus Transaktionen, die als Sale-and-Leaseback geplant waren, aber bilanziell nicht als Verkauf anerkannt wurden.

Vor dem IPO waren das Vermutungen, aber nach dem IPO ist es eine Tabelle in der Bilanz.

OpenAI will diese Tabelle derzeit nicht veröffentlichen.

Die Laufzeiten passen nicht zusammen

Rechenzentren werden über 15 bis 20 Jahre finanziert. Stromanschlüsse, Hallen und Leitungen können so lange halten. Beschleuniger tun es nicht.

Microsoft sagt selbst, daß ungefähr zwei Drittel seines aktuellen Capex in kurzlebige Assets gehen, hauptsächlich GPUs und CPUs. Die Hardware altert in wenigen Jahren. Die Modelle und Inferenzverfahren altern in Monaten.

Auf der Einnahmeseite sind die Laufzeiten teilweise viel kürzer. SpaceX schreibt zum Beispiel, daß seine AI-Cloud-Verträge nach der Anlaufphase mit 90 Tagen Frist gekündigt werden können. In demselben Risikokapitel steht, daß effizientere Modelle und alternative Architekturen die Nachfrage nach Infrastruktur reduzieren können.

Das ist der Kern des Problems:

Die Kredite laufen langsamer ab als die damit beschafften Werte, die den technischen Burggraben bauen sollen, um den Vorsprung zu sichern.

Dazu kommt: Ein Rechenzentrum kann ausgelastet sein und trotzdem eine schlechte Investition werden. Wenn der Marktpreis für dieselbe Aufgabe schneller fällt als Strom, Zins und Abschreibung, erzeugt mehr Nutzung keine gute Marge.

Wir haben dadurch die Situation, daß zugleich die AI-Nutzung explodieren kann, aber AIaaS als Modell nicht ertragreich realisiert werden kann, weil es die erwarteten fantastischen Margen nicht liefern kann. Es generiert höchstens gewöhnliches Hosting mit gewöhnlichen Cloud-Margen.

Moore’s Law für Inferenz

Es gibt kein Moore’s Law für Inferenz . Es gibt keine Industrie-Roadmap, die alle zwei Jahre verläßlich doppelt so viele brauchbare Transistoren zum selben Preis garantiert.

Es gibt aber gerade eine Entwicklung mit derselben wirtschaftlichen Wirkung, die sich vielleicht bis zu einem gewissen Punkt stabilsieren läßt.

Denn alle paar Monate werden Modelle besser, während der Aufwand pro Token sinkt. Der genaue Faktor ist nicht wichtig. Wichtig ist, daß die Faktoren multipliziert werden.

Die Fortschritte kommen aus vielen Richtungen gleichzeitig:

  • kleinere und besser trainierte Dense Models,
  • Quantisierung,
  • Mixture of Experts mit wenigen aktiven Parametern pro Token,
  • kleinere KV-Caches,
  • Gated, Linear und Sparse Attention,
  • Multi-Token Prediction,
  • schnellere Speculative Decoding-Verfahren,
  • mehr Speicherbandbreite durch integriertes LPDDR,
  • getrennte heiße und kalte Modellteile,
  • Streaming selten verwendeter Daten aus RAM und NVMe.

Der letzte Punkt ist neu und wichtig.

Qwen3.8-Flash-Next hat ein Hauptmodell mit 125 Milliarden Parametern, aktiviert aber nur sechs Milliarden pro Token. Dazu kommen weitere 51 Milliarden Parameter als N-Gram-Embedding-Tabelle. Diese Tabelle funktioniert eher wie ein riesiges Dictionary als wie der heiße Kern des Modells. Die Adressen der benötigten Einträge stehen früh fest. Man kann sie aus Host Memory vorladen, während die GPU bereits rechnet. Das bedeutet, während das Modell 90 GB auf der Platte belegt, kommt es mit einem In-Memory Footprint von nur 64 GB, und generiert mit 6 Milliarden aktiven Parametern einen Working Set von einigen GB.

DeepSeek-V4.1-Flash verwendet denselben Grundgedanken mit Engram Conditional Memory: 552 Milliarden Parameter im Backbone, weitere 196 Milliarden in einer dünn angesprochenen Lookup-Struktur und nur acht beziehungsweise 16 Milliarden aktive Parameter für Prefill und Decode.

Solche Tabellen müssen nicht dauerhaft in HBM liegen. Die heißen Teile gehören auf den Beschleuniger, warme Daten in billigeres LPDDR oder normales RAM und kalte, vorhersagbar gelesene Tabellen auf NVMe. Das Modell wird zu einer Speicherhierarchie.

Damit wird aus einem teuren HBM-Problem teilweise ein billiges Lookup-Problem. Das wird ein Standardwerkzeug für LLM-Architekturen werden.

Und die Geschwindigkeit ist mit ( Speicherbandbreite / Anzahl der aktiven Parameter skaliert mit der Quantisierung ) abschätzbar. Ein “A6B” Modell als Q4 hat 3 GB aktiv. Ein M5max mit 634 GB/s kann also circa 210 Token/s generieren, aus einem Modell, das 90 GB auf der Platte belegt.

Low Terra auf einem Mac

Die wirtschaftliche Frist ist keine theoretische Zukunft.

Qwen3.8-27B läuft quantisiert auf einem Mac mit 32 GB Unified Memory gut. Das Modell hat brauchbares Niveau für Software-Entwicklung, es ist vergleichbar einem etwas schlechteren GPT-5.6 Terra (Medium) in dieser Domain. Es ist langsam im Vergleich zu einem Modell aus dem Rechenzentrum mit einem Cluster von H200, aber schnell genug zum Arbeiten.

Ein Terra-class Modell ist für den Specialization Workflow bereits ausreichend. Der normale Workflow verwendet:

  1. Sol mit ‘medium’ für die Entwicklung der User Stories.
  2. Terra mit ‘medium’für die Ableitung der Implementation Tickets.
  3. Luna mit ‘xhigh’ für die Codegenerierung.

Die erste Phase braucht das stärkere Modell (und menschlichen Review), weil dort aus einer unscharfen Idee belastbare Anforderungen entstehen. Fehler an dieser Stelle werden in allen späteren Schritten multipliziert.

Die zweite und dritte Phase sind bereits spezialisiert. Die abgenommenen User Stories beschränken die Tickets. Die abgenommenen Tickets beschränken den Code. Tests, Linter, Type Checker und Reviews liefern mechanische Grenzen.

Für diese beiden Phasen reicht niedriges Terra-Niveau absolut aus. Qwen3.8-27B kann sie lokal erledigen.

Das bedeutet: Zwei der drei Produktionsphasen eines realen Software-Workflows brauchen schon heute keine zentralisierte Inferenz mehr. Gerade dort entsteht aber die Masse der Requests und Token.

Die großen Modelle werden für Architektur, schwierige Reviews und die Verdichtung der Anforderungen gebraucht. Die alltägliche Ticketarbeit und Codeproduktion kann lokal laufen.

Rechner mit 128 GB gemeinsam nutzbarem Speicher sind ebenfalls keine exotischen Server mehr. AMD Ryzen AI Max stellt bis zu 96 GB von 128 GB RAM für die GPU bereit. Nvidias DGX Spark hat 128 GB kohärenten LPDDR5x-Speicher und vier TB NVMe auf dem Schreibtisch. Apple verkauft dieselbe Grundidee als großen Laptop oder Mac Studio.

Auf dieser Klasse läuft heute hohes Terra-Niveau und absehbar niedriges Sol-Niveau, langsamer als im Rechenzentrum, aber lokal, privat und ohne Preis pro Token.

Bessere Modelle erhöhen den Druck

Man könnte annehmen, daß ein neues stärkeres Frontier-Modell den zentralen Anbietern wieder Abstand verschafft. Vielleicht ist das auch so, für manche Anwendungen. Für “SWE” (Software Engineering) ist was wir haben aber in der Regel ausreichend.

Und der Druck wird größer:

Wenn wir alle drei Monate die Leistung verdoppeln und zugleich den Inferenzaufwand halbieren, ist das wirtschaftlich eine Vervierfachung. Faktor 100 sind nicht hundert Generationen. Es sind mehr als drei und weniger als vier solche Schritte:

4³ = 64
4⁴ = 256

Ob jeder einzelne Schritt exakt nach drei Monaten kommt ist nicht wichtig. Die heutigen Rechenzentren sind auf Laufzeiten von bis zu 19 Jahren finanziert. Selbst eine deutlich langsamere Verbesserung gewinnt dieses Rennen.

Es wird keine Pause geben

Eine koordinierte Pause wäre für die westlichen Anbieter ideal.

OpenAI, Anthropic, Google, Meta und xAI könnten den Frontier-Wettlauf verlangsamen. Sie würden weniger neues Capex brauchen, die Lebensdauer der vorhandenen Modelle verlängern und die Preise länger oben halten.

Das ist ein klassisches Abrüstungsproblem. Jeder einzelne Anbieter muß weiter investieren, weil ein anderer sonst die Capability Leadership übernimmt. Gemeinsam wären sie finanziell besser gestellt, wenn alle weniger investierten.

Also nennt man die gewünschte Koordination AI Safety.

Das Problem ist: China nimmt an diesem Abrüstungsabkommen nicht teil. DeepSeek, Qwen, GLM, Kimi, MiMo und ihre Nachfolger werden weiterentwickelt. Open Weights werden veröffentlicht, kopiert, quantisiert, destilliert und auf neue Hardware portiert.

Ein veröffentlichtes Modell kann man nicht zurückrufen. Es liegt auf privaten Rechnern, in Firmenarchiven und auf Rechnern außerhalb der USA. Selbst wenn morgen alle westlichen Labs stillstehen, arbeitet die bereits veröffentlichte Technik weiter.

Trump hat eine Pause zunächst mit dem korrekten Argument abgelehnt, daß die USA damit den Vorsprung an China abgeben würden. Die Pause wird die amerikanische Dominanz schwächen: mehr nicht-amerikanische und nicht-kontrollierte AI.

Regulatory Capture

Wenn die technische Pause nicht funktioniert, bleibt eine regulatorische.

Das Ziel ist nicht unbedingt ein ausdrückliches Besitzverbot für leistungsfähige Rechner. Das wäre auffällig, schwer durchsetzbar und schnell technisch veraltet. Es reicht, den gewerblichen Einsatz lokaler und offener Modelle unpraktikabel zu machen.

Die Werkzeuge dafür existieren:

  • Zertifizierungs- und Auditpflichten für Modelle und Betreiber,
  • Registrierung großer Modelle und Compute-Installationen,
  • verpflichtendes Logging und zentralisierte Überwachung,
  • persönliche Haftung von Geschäftsführern,
  • Versicherungspflichten,
  • Cloud-KYC und Herkunftsnachweise für Modelle,
  • Procurement-Regeln für Staat und regulierte Branchen,
  • Ausschluß nicht zertifizierter Open Weights,
  • Exportkontrollen auf Beschleuniger, Speicher und Model Weights,
  • Einschränkungen gegen chinesische Modelle aus Gründen der Nationalen Sicherheit.

Privat darf man das Modell dann vielleicht noch besitzen. Eine Bank, Versicherung, Behörde, Arztpraxis oder größere Firma darf es nicht mehr einsetzen. Für sie bleibt nur ein zertifizierter Managed Service.

Damit wird aus AI Safety ein Schutzwall für AIaaS.

Das Cornern von GPU-, RAM- und NVMe-Märkten ergänzt diesen Schutzwall. Wer Rechenzentren für Jahre im voraus mit Beschleunigern, HBM, DRAM und Flash versorgt, verteuert gleichzeitig die lokale Konkurrenz. Der Effekt ist bereits sichtbar: leistungsfähige GPUs, große RAM-Ausstattungen und schnelle NVMe sind knapp und teuer.

Der Gegeneffekt ist auch bereits sichtbar: CXML und andere chinesische Anbieter machen sich bereit, die so entstande Nachfrage zu befriedigen.

Am Ende hält all das die Entwicklung nicht auf. Es verschiebt sie zu Architekturen, die weniger von jedem knappen Gut brauchen. MoE, Quantisierung, kleine KV-Caches, LPDDR und NVMe-Streaming sind genau die Antwort auf einen gecornerten Hardwaremarkt. Das bedeutet: Wir bekommen auf jeden Fall mehr und leistungsfähigere Modelle, die optimiert darauf sind, auf lokaler Hardware befriedigende Leistung zu bringen.

Wenn es aber um regulatory capture geht, dann kann die EU dabei als williger Marktvernichter dienen. Je komplexer Registrierung, Haftung und Zertifizierung werden, desto weniger europäische Firmen können eigene Modelle betreiben. US-Hyperscaler können die Compliance-Kosten auf Millionen Kunden verteilen. Ein Mittelständler mit einem offenen Modell kann das nicht.

Die Regulierung schützt dann nicht Europa vor amerikanischer AI. Sie schützt amerikanische AI vor europäischer Konkurrenz.

Der Exit

Für die AI-Investitionen gibt es nur wenige mögliche Auswege aus dem aktuellen Szenario:

Ein reguliertes Oligopol kann die Preise hochhalten und die vorhandenen Verpflichtungen auf die Kunden umlegen.

Open Weights können AI kommoditisieren. Dann überleben Rechenzentren und Hosting, aber die außergewöhnlichen AIaaS-Margen verschwinden.

Ein großer Kunde kann ausfallen oder Verträge reduzieren. Dann beginnt der Kreditunfall bei CoreWeave oder einem Oracle-Projekt und die Ansteckung läuft durch Zweckgesellschaften, Leasingfirmen, Banken und Private Credit.

Die Hyperscaler können ihre Fehlallokationen abschreiben. Ihre Werbe-, Office- und Cloud-Geschäfte werden das überleben. Aktionäre und Mitarbeiter bezahlen die Rechnung.

Oder der Staat sichert die Investitionen indirekt: mit Kreditgarantien, Stromverträgen, beschleunigten Genehmigungen, Rüstungs- und Behördenaufträgen und einer Regulierung, die Nachfrage nach den zugelassenen Anbietern erzwingt.

Eine formelle Verstaatlichung ist dafür nicht nötig. Gewinne bleiben privat. Das Risiko kann man auch so sozialisieren.

Die Modelle werden billiger, die Schulden nicht

Der Börsengang würde zeigen, wie schlecht die Unit Economics der Frontier-Anbieter tatsächlich sind. Deshalb wird er verschoben.

Die Pause würde den Capex-Wettlauf bremsen, die vorhandenen Modelle länger verwertbar machen und Zeit für regulatorische Schutzwälle kaufen. Deshalb wird sie verlangt.

Aber die Pause wird nicht stattfinden.

China entwickelt weiter. Open Weights sind bereits publiziert. Architekturen werden effizienter. 32-GB-Rechner erledigen heute Arbeiten, für die vor kurzem noch ein proprietärer Cloud-Service notwendig war. 128-GB-Rechner bringen die nächste Modellklasse auf den Schreibtisch.

Die Branche hat Rechenzentren für bis zu 19 Jahre bestellt, aber ihr Vorsprung wird in Quartalen gemessen.

Sie wollen eine Pause. Es wird keine geben.

Top

Die Menschmaschine muß eingearbeitet werden

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

Der Real-SWE-Benchmark wirft moderne Coding Agents auf private Codebasen realer Firmen und gibt ihnen Aufgaben, die dort tatsächlich angefallen sind. Das Ergebnis sieht im ersten Moment schlecht aus: Der beste getestete Agent löst 38,8 Prozent, GPT-6 Astra mit Codex 33,8 Prozent und GPT-5.6 Sol mit Codex 16,2 Prozent.

Genau deshalb halte ich das Ergebnis für unerwartet gut. Die Zahlen sind erstaunlich hoch.

Der Benchmark setzt einen neuen Mitarbeiter ohne Einarbeitung vor eine proprietäre Enterprise-Codebase, gibt ihm ein Ticket mit median 1.742 Zeichen und erwartet eine fertige Änderung. Die Referenzlösung berührt median elf Dateien. Es geht um Abrechnung, Steuern, Migrationen, Identitäten und verteilte Datenspeicher. Das sind Aufgaben, bei denen eine plausible Lösung nicht genügt. Sie muß zu den Geschäftsregeln und Arbeitsweisen genau dieser Firma passen.

Wenn ein Uni-Abgänger oder Postdoc bei seinem ersten Tag in einer Firma aus einem solchen Ticket in einem Versuch eine korrekte produktionsreife Änderung baut, hat man großes Glück gehabt. Normalerweise stellt er Fragen, liest Dokumentation, spricht mit Kollegen, schreibt einen Plan, läßt ihn prüfen, implementiert in Schritten und bekommt seinen Code im Review zurück.

Warum sollte ein LLM ohne all das besser abschneiden?

Resolution Rate im Real-SWE-Benchmark. Gezeigt wird Pass@1, gemittelt über acht unabhängige Läufe pro Aufgabe; die Striche zeigen 95-Prozent-Konfidenzintervalle. Quelle: Specific Labs, Real-SWE .

Dieser Text basiert auf einem Mastodon-Thread über Firmenwissen und LLMs .

Zwei Sorten Wissen

Eine Hochschule kann allgemeines Wissen über ein Fach vermitteln. Das sind Grundlagen, Methoden, Begriffe und der aktuelle Stand der Technik. Es ist absichtlich austauschbar: Buchhaltung, Werkstoffkunde, Vertragsrecht, Softwareentwicklung oder Statistik werden nicht für genau eine Firma erfunden. Wer gut ausgebildet ist, kennt nicht jede Antwort, kann aber lernen, bewerten und eine begründete Antwort erarbeiten.

Eine Firma besitzt zusätzlich lokales Wissen. Wie man dieses Wissen nennt, ist weniger wichtig als sein Inhalt. Es beschreibt, warum dieser Laden diese Dinge in dieser Form macht:

  • welche Kunden welchen Sonderfall brauchen;
  • welche Risiken in der Vergangenheit tatsächlich eingetreten sind;
  • welche Entscheidung ein Review oder ein ausdrückliches Go verlangt;
  • welche Systeme nur scheinbar unabhängig sind;
  • welche Abkürzung einmal teuer geworden ist;
  • welche Qualität die Firma verspricht und worin ihr Alleinstellungsmerkmal besteht.
  • und natürlich die Dinge, die nur diese Firma kann und umsetzt, und die deswegen ein Alleinstellungsmerkmal sind.

Dieses Wissen kann nicht vollständig an der Universität gelehrt werden. Wäre es allgemeingültig und überall verfügbar, wäre es kein Alleinstellungsmerkmal. Eine Firma muß allgemeines Fachwissen deshalb durch in ihr lokales Firmenwissen anreichern.

Das geschieht durch Einarbeitung, Zusammenarbeit, Prozesse und Reviews. Neue Mitarbeiter hören Gespräche, beobachten Entscheidungen, stellen Fragen und lernen, welche geschriebenen Regeln wörtlich gelten und wo noch eine unsichtbare Nebenbedingung existiert. Sie eignen sich in der Kantine, im Meeting und im Code Review Dinge an, die nirgendwo sauber dokumentiert sind.

Ein LLM kann das nicht. Es hört nicht zu und bekommt nichts zufällig mit. Es kann lesen und schreiben. Wenn das Firmenwissen nicht in lesbarer Form existiert, existiert es für das Modell nicht.

Verschriftlichung tut weh

Viele Firmen sind schlecht darin, ihre eigene Arbeit auf der Metaebene zu beschreiben. Sie können sagen, was sie seit zwanzig Jahren tun. Schwieriger ist die Erklärung, worin heute die Wertschöpfung besteht, warum ein Schritt vor dem nächsten kommt und welche Erkenntnis zu einer bestimmten Regel geführt hat.

Ein Teil davon ist gewöhnliche Betriebsblindheit. Wer eine Arbeit lange allein erledigt, trägt einen Haufen unverbalisierter Annahmen im Kopf. Man ahnt, daß etwas falsch ist, ohne die Kriterien noch benennen zu können. Das funktioniert für eine Person, solange sie verfügbar ist.

Beim Wachsen muß dieser Dunstgeist aus dem Kopf heraus. Aus einer Person werden nicht einfach zwei unabhängige Personen. Sobald mehrere Menschen dieselbe Arbeit teilen, muß jemand Zustände, Übergaben und Entscheidungen koordinieren. In der Gegend von 15 und später noch einmal von 150 Menschen kommen typische Sprungstellen: Gründer und Prozeßeigner müssen implizite Kontrolle abgeben, Regeln erklären und Entscheidungen reproduzierbar machen. Wenn sie das nicht schaffen, wächst der Laden nicht weiter oder zerlegt sich beim Versuch.

Manchmal ist die fehlende Transparenz auch Absicht. Sobald eine Regel mit Grund und Ziel aufgeschrieben ist, wird sie diskutierbar und veränderbar. Nicht jeder, der von einer undokumentierten Entscheidung Macht bezieht, will das.

LLMs machen dieses Problem sichtbar. Für ihren Einsatz muß die Firma ihr lokales Wissen externalisieren, konkretisieren und linearisieren. Sie muß es also aufschreiben. Das tut weh, weil dabei nicht nur Text entsteht. Es werden Widersprüche, unbegründete Rituale, private Fürstentümer und nie entschiedene Sonderfälle sichtbar.

Der Lacher ist: Zertifizierung und Compliance haben viele Firmen bereits gezwungen, genau diese Arbeit wenigstens teilweise zu erledigen. Der angeblich unsinnige bürokratische Overhead dokumentiert Rollen, Freigaben, Kontrollen und Nachweise. Schlechte Compliance produziert Papierfriedhöfe. Gute Compliance erzeugt eine lesbare Beschreibung, wie der Laden zuverlässig arbeitet. Das ist auch die Grundlage für den Einsatz eines LLM.

Ein Prozeß ist Kommunikation

Für die Menschen innerhalb eines schlechten Prozesses sieht ein Prozeß wie eine Arbeitsverhinderungsmaßnahme aus. Formulare, Tickets, Freigaben und Meetings bremsen die eigentliche Arbeit und behindern alle mit Bürokratie.

Das ist oft korrekt beobachtet, aber nicht der Sinn eines Prozesses.

Ein brauchbarer Prozeß ist eine Arbeits- und Kommunikationsanweisung. Er legt fest, wer wann warum mit wem worüber sprechen muß, welche Information dabei entsteht und unter welcher Bedingung die Arbeit in den nächsten Zustand wechselt. Sein Ziel ist Reproduzierbarkeit: Das Ergebnis soll nicht davon abhängen, daß eine bestimmte Person zufällig im Raum war und noch wußte, was 2019 schiefging.

Reviews sind ein Teil davon. Sie sind keine Zeremonie, sondern Stellen, an denen fehlendes Wissen ergänzt und falsche Annahmen gestoppt werden. Manche Reviews müssen Menschen machen. Andere lassen sich als mechanische Gates ausdrücken: Zahlen müssen sich ausgleichen, Pflichtfelder müssen vorhanden sein, eine Checkliste muß vollständig sein oder ein Test muß grün werden.

Genau solche Prozesse braucht auch ein LLM. Nicht weil es eine besonders dumme Maschine ist, sondern weil jede neue Arbeitskraft ohne lokales Wissen sie braucht.

Kontext ist Kurzzeitgedächtnis

Ein LLM hat bei der Arbeit keinen dauerhaften inneren Zustand. Als vereinfachtes Modell kann man es sich wie einen REST-Service vorstellen: Ein Request geht hinein, eine Antwort kommt heraus. Der nächste Request enthält die Vorgeschichte erneut. Das Modell erinnert sich nicht an die Unterhaltung. Die Software um das Modell, der Harness, schickt sie wieder mit.

Diese mitgeschickte Arbeitsmenge heißt Kontext. Darin stehen die Anweisungen, die bisherige Unterhaltung, gelesene Dokumente und Ergebnisse aus Werkzeugen. Der Kontext ist das Kurzzeitgedächtnis der aktuellen Arbeit.

Er hat eine maximale Größe, das Context Window. Moderne Modelle können sehr große Fenster anbieten, zum Teil bis in die Größenordnung von einer Million Token. Das heißt nicht, daß sie bis zum letzten Token gleich gut arbeiten. Mit wachsendem Kontext konkurrieren relevante Regeln mit alten Antworten, langen Logs und inzwischen bedeutungslosen Zwischenergebnissen um Aufmerksamkeit.

Matt Pocock nennt den späteren Bereich die „Dumb Zone“ . Bei aktuellen Modellen beobachtet er den Übergang oft ungefähr zwischen 125.000 und 150.000 Token, aber das ist keine harte Grenze. Modell, Aufgabe und Qualität des Kontextes verschieben sie. Der Punkt ist unabhängig von der genauen Zahl: Ein großes technisches Fenster ist keine Erlaubnis, den gesamten Speicherboden da hineinzurümpeln.

Wenn das Modell Regeln vergißt, frühere Fehler wiederholt und trotz vorhandener Information zu raten beginnt, hilft nicht unbedingt noch mehr Text. Dann beginnt man mit einem frischen Kontext und lädt gezielt das wieder ein, was für den nächsten Arbeitsgang gebraucht wird.

Dateien sind Langzeitgedächtnis

Ein Coding Harness gibt dem Modell Werkzeuge. Es kann Dateien im aktuellen Repository lesen und schreiben, Programme ausführen, Tests starten und Änderungen mit Git vergleichen. Dadurch bekommt die Arbeit neben dem Kurzzeitgedächtnis ein Langzeitgedächtnis: die Dateien im Repo.

Das Modell kennt diese Dateien nicht magisch. Es muß sie finden und lesen. Dafür braucht es Indizes: ein README, eine AGENTS.md oder eine andere kurze Karte, in der steht, welches Wissen wo liegt und wann es relevant ist. Der Index gehört früh in den Kontext. Die großen Detaildokumente werden nur geladen, wenn die aktuelle Aufgabe sie braucht.

Nicht jedes Firmenwissen liegt in Markdown neben dem Code. Tickets stehen in Jira, Entscheidungen in Confluence, Kundendaten in Datenbanken und Kennzahlen in internen Diensten. MCP-Server und andere Integrationen können solche Quellen für den Harness zugänglich machen. Auch dort braucht man Beschreibungen und Suchwege. Eine Datenhalde wird nicht dadurch zu Wissen, daß das Modell theoretisch darauf zugreifen darf.

Das Modell kann außerdem Ergebnisse aus seinem Kurzzeitgedächtnis in Dateien schreiben: eine geklärte Anforderung, einen Plan, offene Fragen oder ein Review. Im nächsten frischen Kontext liest es diese Artefakte wieder ein. So überlebt das Ergebnis eines Arbeitsganges, ohne daß die gesamte Unterhaltung erhalten bleiben muß.

Skills sind Arbeitsanweisungen

Ein Skill ist keine neue Intelligenz und kein magischer Experte im Modell. Er ist eine kleine, wiederverwendbare Arbeitsanweisung: Verwende für diese Sorte Aufgabe diese Schritte, lies diese Quellen, erzeuge dieses Format und prüfe das Ergebnis auf diese Weise.

Das lokale Firmenwissen steckt also nicht vollständig im Skill. Der Skill erklärt, wie es gefunden und angewendet wird. Ein TDD-Skill kann zum Beispiel verlangen:

  1. beobachtbares Verhalten festlegen;
  2. einen Test schreiben, der das fehlende Verhalten zeigt;
  3. nur den kleinsten Code schreiben, der den Test bestehen läßt;
  4. erst bei grünem Test refaktorieren;
  5. vor der Übergabe die Projektprüfungen ausführen.

Das ist keine Garantie für den richtigen Geschäftsvorgang. Der Skill verhindert aber, daß der Agent ohne prüfbare Grenze einfach plausiblen Code erzeugt. Die geschäftliche Anforderung kommt aus den Firmendokumenten; der Skill liefert die Arbeitsdisziplin.

One-Shotting testet den falschen Arbeitsmodus

Real-SWE verwendet kurze, leicht unterspezifizierte Aufgaben. Die Details sollen aus der Codebase und den angeschlossenen Werkzeugen entdeckt werden. Das ist als Benchmark legitim und interessant. Es mißt aber einen ersten Lauf. Die Resolution Rate ist Pass@1.

Die häufigsten Fehler passen exakt zum Einarbeitungsproblem: Anforderungen werden übersehen, Annahmen nicht geprüft und richtige Ideen falsch in das vorhandene System integriert. Das sind keine exotischen Maschinenfehler. Das sind die Fehler eines Neulings, der zu früh losarbeitet.

In realer Arbeit würde ich aus einem 1.700-Zeichen-Ticket nicht sofort Code erzeugen lassen. Der erste Arbeitsgang macht daraus strukturierte User Stories. Das Modell muß dabei ausdrücklich „Offene Fragen“ und „Wichtige Hinweise“ liefern. Dort landen die Annahmen, die der Ausgangstext nicht auflösen konnte.

Die User Stories werden geprüft und korrigiert. Danach zerlegt ein weiterer Arbeitsgang sie in kleine Implementation Tickets, wieder mit Review. Erst dann wird Ticket für Ticket implementiert, jeweils mit Test, kleinstem Code, Projektprüfungen und einem Git-Checkpoint.

Das ist der Spezialisierungs-Workflow :

  1. Aus einem groben Ziel werden explizite User Stories.
  2. Reviews ergänzen Sicherheit, Architektur und lokale Regeln.
  3. Aus den abgenommenen Stories werden geordnete, testbare Tickets.
  4. Aus jedem Ticket wird in einem frischen, begrenzten Kontext Code.
  5. Tests, Reviews und Git machen Fehler sichtbar und Schritte rückgängig.

Ein konkretes Beispiel sind die User Stories und die daraus erzeugten Implementation Tickets für den server-key-injection-Tracer.

Jedes Arbeitsergebnis ist nie besser als sein Briefing. Ein gutes Briefing ist dabei nicht notwendig ein riesiger Prompt. Es ist die Kombination aus einer klaren Aufgabe, auffindbarem Firmenwissen, einer passenden Arbeitsanweisung und den richtigen Kontrollen.

Die positive Lesart des Benchmarks

Real-SWE zeigt nicht, daß LLMs für Enterprise-Arbeit unbrauchbar sind. Der Benchmark zeigt, daß allgemeines Wissen lokales Firmenwissen nicht ersetzt und daß One-Shotting für komplexe Arbeit nicht genügt. Das war vorher bei Menschen so und ist bei Modellen nicht anders.

Wer Resultate will, muß:

  • die eigene Arbeitsweise explizit machen;
  • das lokale Wissen verschriftlichen und erschließen;
  • Aufgaben in überprüfbare Arbeitsgänge zerlegen;
  • an den richtigen Stellen menschliche Reviews und mechanische Gates setzen;
  • früh im Spezialisierungs-Workflow steuernd eingreifen;
  • neue Kontexte beginnen, bevor der alte in die Dumb Zone rutscht.

Das ist anstrengender als ein langer Prompt und billiger als eine Serie falscher produktiver Änderungen. Es verbessert außerdem nicht nur die Arbeit mit LLMs. Eine Firma, die erklären kann, wie sie Wert erzeugt, Entscheidungen trifft und Wissen findet, kann auch Menschen besser einarbeiten, Vertretungen organisieren und Prozesse verändern.

LLMs arbeiten dabei überraschend menschlich. Sie lernen nicht während des Einsatzes wie ein Mensch, aber ihre Fehler werden mit denselben Mitteln beherrschbar: Einarbeitung, kleine Aufgaben, sichtbare Zwischenergebnisse, Rückfragen, Reviews und überprüfbare Grenzen.

Wir haben LLMs in unserem Angesicht geschaffen und mit unseren Schriften trainiert. Nun arbeiten sie wie wir und machen Fehler wie wir. Diese Fehler lassen sich erfassen und auf dieselbe Weise korrigieren wie unsere.

Eventuell klappt das nicht mit dem Maschinengott. Aber wir bauen gerade die Menschmaschine.

Top

Lage, Lage, Lage – und der fossile Fantasiepreis

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

Lage, Lage, Lage. Das ist beim Hauskauf immer noch richtig, aber es ist halt nicht alles.

Die Lage entscheidet, ob man dort leben will. Der Zustand des Gebäudes entscheidet, wie viel Geld man nach dem Kauf noch hineinwerfen muß, damit man dort auch leben kann. Im Jahr 2026 gehören dazu nicht nur Dach, Keller und Fenster. Ein Haus muß elektrisch heizbar sein, genug Strom bekommen, Fläche für Photovoltaik haben und am Glasfasernetz hängen. Fehlt das, ist das nicht zwingend ein Beinbruch, aber fast sicher eine Verzögerung und ein Kostenfaktor, den man einpreisen muß.

Der Angebotspreis ist dann ein fossiler Fantasiepreis. Bezahlt wird der Wert nach Sanierung, abzüglich der Kosten, der Risiken und der Monate, in denen man wegen der Arbeiten noch nicht einziehen kann.

Dieser Text basiert auf einem Mastodon-Thread zum Hauskauf .

Erst filtern, dann verlieben

Beim Hauskauf ist das Angebot groß, unübersichtlich und voller schlecht dokumentierter Objekte. Bei uns waren es in Deutschland und später noch einmal in den Niederlanden jeweils einige tausend Anzeigen. Die kann man nicht alle besichtigen. Man braucht harte Filter.

  • Kein Grundriß? Weg.
  • Energieverbrauch über 75 kWh/(m²·a)? Weg oder Sanierung vollständig einpreisen.
  • Keine Glasfaser? Weg.
  • Öffi-Anbindung Dreck? Weg.
  • Kein brauchbarer dreiphasiger Stromanschluß? Weg.
  • Dach nicht für PV geeignet? Weg oder den Minderwert berechnen.
  • Keine realistische Möglichkeit für Wärmepumpe und Ladeanschluß? Weg oder Umbau einplanen.

Diese Liste ist absichtlich brutal. Man prüft damit nicht, ob ein Haus schön ist, sondern ob es die Mindestbedingungen erfüllt. Schönheit ist billig zu besichtigen und teuer zu vergessen. Ein ungeeignetes Energiesystem, eine schlechte Verkehrslage oder fehlende Infrastruktur begleiten einen dagegen jeden Tag.

75 kWh/(m²·a) ist dabei ein guter Suchfilter, kein Naturgesetz. Spätestens oberhalb von 100 kWh/(m²·a) ist die Hülle schwer mit einer Wärmepumpe zu beheizen und eine energetische Investition gehört zwingend in die Rechnung. Den Wert bekommt man aus dem Energieausweis, besser noch aus dem realen Verbrauch der letzten drei Winter. Ein einzelner milder Winter oder die Gewohnheiten eines sparsamen Vorbesitzers sind keine belastbare Grundlage. Drei Jahre zeigen wenigstens die Größenordnung.

Fossil ist vorbei

Eine Gasheizung ist 2026 kein Vermögenswert. Sie ist eine Entsorgungs- und Umbauposition.

Die Versorgung mit fossilen Brennstoffen ist politisch unsicher, teuer und im Preis steigend. Langfristig ist sie klimatisch untragbar. Wer heute viel Geld für ein Haus ausgibt und dessen dauerhafte Nutzbarkeit an Gas oder Öl bindet, kauft sich ein absehbares zweites Projekt dazu.

Das bedeutet nicht, daß jedes Haus mit Gasheizung unbewohnbar ist. Es bedeutet, daß die Gasheizung den Kaufpreis nicht erhöht. Ihr Ersatz und alle dafür nötigen Nebenarbeiten mindern den heutigen Wert des Hauses. Fehlt eine Wärmepumpe, muß man prüfen, ob sie installiert werden kann, welche Heizflächen sie braucht und wie weit dafür Vorlauf, Leitungen, Warmwasserbereitung und Sicherungskasten umgebaut werden müssen.

Die Dämmung bestimmt dabei den Heizbedarf. Große Heizflächen und niedrige Vorlauftemperaturen bestimmen, wie effizient eine Luft-Wasser-Wärmepumpe arbeiten kann. Luft-Luft-Wärmepumpen können in passenden Gebäuden eine Alternative sein. Aber irgendeine elektrische Antwort auf die Heizungsfrage muß das Haus haben. „Der Brenner läuft doch noch“ ist keine Antwort, sondern die Beschreibung des Problems.

Strom muß liegen

Strom ist nicht nur Licht und Steckdose. Strom ist Heizung, Warmwasser, Auto, Küche, Server, Batterie und der Rest des Hauses.

Am Gebäude müssen drei Phasen mit ausreichender Absicherung anliegen. 3 × 25 A sind eine sinnvolle Mindestgröße – passend für einen Ladeanschluß und eine Kochplatte, bis das Auto abregelt, weil der Anschluß nicht mehr kann. In Deutschland findet man meist 3 × 63 A, dann ist alles gut. Im Ausland ist die Situation meist komplizierter und beschränkter.

Der Sicherungskasten muß auf aktuellem Stand sein und genügend Platz für Wärmepumpe, PV, Batterie und Ladeanschluß bieten. Wenn das alles neu muß, ist das lösbar. Es ist nur nicht kostenlos.

Wenn man die Elektrik ohnehin öffnet, macht man sie einmal richtig – der Bestandschutz entfällt dann sowieso. In unserer Berliner Altbauwohnung hieß das: eine Sicherung pro Raum, FI-Schutz, an jeder Ecke eine Steckdose und aus einzelnen vorhandenen Steckdosen Dreiergruppen machen. Dazu zwei Netzwerkkabel in jeden Raum, zentral auf ein Patchfeld geführt. So lassen sich Access Points, DECT-Antennen und Arbeitsplätze dort anschließen, wo sie gebraucht werden. In einer langen Wohnung mit zwei internen Brandmauern hat sich das über Jahre bezahlt gemacht.

Der Anlaß für diese Sanierung war übrigens nicht ästhetisch. Am Sicherungskasten roch es komisch und dann war es knapp: Ein Sicherungsautomat hatte angefangen zu schmoren. Erst kam die Notinspektion, danach alles neu. Alte Elektrik bleibt nicht deshalb gut, weil sie bisher nicht gebrannt hat.

Eigenleistung kann die Rechnung verbessern, wenn man Geld gegen eigene Zeit tauschen kann. Die Rechnung ist einfach: Ist der eigene Stundenlohn niedriger als der des Handwerkers, kann man die Arbeit fachlich sauber ausführen, und macht sie einem wenigstens ein bißchen Spaß? Wenn nicht, bezahlt man Menschen, die das können, und ist nett zu ihnen.

PV braucht Fläche, Internet braucht ein Kabel

Strom wird mit eigener PV billiger. Dafür muß geeignete Dach- oder Fassadenfläche vorhanden sein. Ein Süddach ist nicht die einzige brauchbare Lösung. Ost-West liefert weniger Jahresertrag pro installiertem Modul, verteilt die Leistung aber besser auf Morgen und Abend, wenn im Haus tatsächlich Verbrauch stattfindet. Eine Batterie kann den Rest vom Mittag in den Abend verschieben.

Auch hier gilt: Fehlt die Anlage, ist das kein Ausschlußgrund, wenn sie technisch nachrüstbar ist. Dann steht sie nur auf der Investitionsliste und wird vom Kaufpreis abgezogen. Ist das Dach verschattet, statisch ungeeignet oder durch Denkmalschutz praktisch blockiert, ist die Lage schlechter.

Und dann ist da das Internet. Glasfaser, sonst zählt es halt eher nicht.

Okay, man kann mit 100/100 überleben, wenn man muß, aber das ist nicht das Ziel: Ein Haus kauft man für Jahrzehnte. Kupfer wird in dieser Zeit nicht jünger und ein Mobilfunkvertrag macht aus einem Gebäude keinen verläßlichen Arbeitsort. Ob man heute 1 GBit/s auslastet, ist nicht die Frage. Entscheidend ist, ob eine moderne, aufrüstbare Leitung am Haus liegt. Bei der Elektrik akzeptiert auch niemand als Zukunftskonzept, daß die eine Verlängerungsschnur im Wohnzimmer doch bisher gereicht hat.

Eine Eigentümergemeinschaft ist eine Schicksalsgemeinschaft

Eigentumswohnungen haben oft weniger Außenfläche als freistehende Häuser. Kein eigenes Dach und nur zwei Außenwände können den Heizbedarf senken. Wohnungen liegen außerdem häufiger zentral, so daß gute öffentliche Verkehrsmittel den eigenen Ladeanschluß weniger wichtig machen.

Dafür gehört einem das Gebäude nicht allein. Dach, Fassade, Heizung, Leitungen und Ladeinfrastruktur hängen an der Eigentümergemeinschaft. Man kauft deshalb nicht nur Quadratmeter, sondern für Jahrzehnte eine Schicksalsgemeinschaft.

Die Protokolle der Eigentümerversammlungen der letzten fünf Jahre sind Pflichtlektüre. Dort steht, ob notwendige Arbeiten beschlossen, verschoben oder regelmäßig zerredet werden. Wohnen die Eigentümer selbst im Haus oder ist es nur Kapitalanlage? Wird investiert oder abgewohnt? Gibt es genügend Rücklagen? Leben die Leute in der realen Welt oder wollen sie den Zustand von 1987 konservieren?

Wenn die Gemeinschaft nicht paßt, paßt die Wohnung nicht.

Besichtigen lassen, nicht nur besichtigen

Nach den harten Filtern bleiben vielleicht zehn Objekte übrig. Dort geht man mit Freunden durch, die etwas vom Handwerk verstehen und die Bude dissen sollen. Ihre Aufgabe ist nicht, die hohen Decken zu bewundern. Sie sollen zeigen, wo Arbeit, Geld und Zeit verschwinden werden.

Vor irgendeinem Commitment kommt ein unabhängiger Gutachter. Das kostet Geld und ist genau deshalb gut angelegtes Geld. Ein übersehener feuchter Keller, ein beschädigtes Dach oder eine unbrauchbare Installation kostet mehr.

Die Sanierung wird großzügig gerechnet. Dazu kommen Reserven für das, was erst nach dem Öffnen von Wand, Boden oder Dach sichtbar wird. Bestandshäuser kosten mehr und dauern länger. Altbauten erst recht.

Und der Einzug wird nicht auf den Tag nach dem Notartermin geplant. Drei bis sechs Monate zusätzliche Miete gehören in die Kalkulation. In einem leeren Haus lassen sich Elektrik, Heizung, Böden und Wände schneller und billiger machen. Zwischen Kartons, Möbeln und Bewohnern wird jede Baustelle zum Hindernislauf.

Der Kaufpreis ist eine Rechnung

Der Verkäufer kann verlangen, was er will. Daraus entsteht noch kein Wert.

Man nimmt den Preis eines technisch brauchbaren, bezugsfertigen Hauses und zieht ab:

  • Wärmepumpe und Warmwasserbereitung;
  • Dämmung, Fenster und Heizflächen;
  • Sicherungskasten, Leitungen und Netzanschluß;
  • PV, Batterie und Ladeanschluß;
  • Dach, Feuchtigkeit und sonstige Bauschäden;
  • Gutachter, Planung und Risikopuffer;
  • drei bis sechs zusätzliche Monatsmieten.

Dann hat man einen verhandelbaren Kaufpreis. Hat der Verkäufer die werterhaltenden Arbeiten nicht gemacht, ist das sein Problem. Er bringt unheizbaren, stromlosen Schrott auf den Markt. Der Käufer muß dessen Sanierung nicht zweimal bezahlen: einmal im Fantasiepreis und danach noch einmal beim Handwerker.

Grundbuch, Plan und Genehmigungen

In Deutschland genießt das Grundbuch öffentlichen Glauben. Das ist Juristensprache für “Beim Erwerb gilt sein Inhalt grundsätzlich als richtig, solange kein Widerspruch eingetragen und dem Käufer die Unrichtigkeit nicht bekannt ist (§ 892 BGB ).”

Also braucht man einen aktuellen Grundbuchauszug. Dort stehen Eigentum, Grundpfandrechte, Dienstbarkeiten und andere Belastungen. Alte Grundschulden müssen nicht automatisch ein Problem sein, aber ihre Löschung oder Übernahme muß geklärt sein, weil die finanzierende Bank ihren eigenen Rang abgesichert haben will.

Dazu kommen Bebauungsplan, Baulastenverzeichnis und Baugenehmigungen. Der ausgebaute Dachboden, der Anbau und die Garage werden nicht dadurch legal, daß sie schon lange stehen. Die Bank prüft einen Teil davon aus Eigeninteresse. Bis zu einem Punkt ist sie deshalb Verbündeter: Sie will ebenfalls kein unverkäufliches Pfand finanzieren. Die Prüfung ersetzt trotzdem nicht den eigenen Gutachter und die eigenen Unterlagen.

Der Kredit muß Luft lassen

Eigenkapital hilft. Sehr.

Man kann bei der Finanzierung hart am Wind segeln, aber man schreibt das nicht als Pflicht in den Vertrag. Niemand weiß, was mit Job, Gesundheit, Familie oder Haus passiert. Eine hohe zwingende Rate macht aus einem schlechten Jahr eine Krise.

Wir haben einen langfristigen Vertrag mit kleiner regulärer Tilgung gewählt und dafür eine jährliche Sondertilgung von bis zu zehn Prozent hineinverhandelt. Der Job lief gut, es gab Boni, und wir konnten schneller tilgen. Wäre es anders gelaufen, hätte uns der Vertrag nicht sofort erwürgt.

Banken sind Geschäfte. Zinsbindung, Tilgungssatz, Sondertilgung und andere Konditionen sind Produkteigenschaften und verhandelbar. Man vergleicht Angebote und läßt sich nicht vom ersten freundlichen Menschen mit Taschenrechner beeindrucken.

Schulden sind nicht automatisch schlecht. Sie müssen zum Einkommen, zum Risiko und zur Nutzung der Immobilie passen. Bei einer selbstbewohnten Immobilie senken normale Kreditzinsen nicht einfach das zu versteuernde Gehalt. Bei einer vermieteten Immobilie können Schuldzinsen als Werbungskosten wirken, soweit sie wirtschaftlich mit den Mieteinkünften zusammenhängen (§ 9 EStG ); die Tilgung selbst ist keine Werbungsausgabe. Dafür gibt es unter anderem die Abschreibung des Gebäudes. Wer daraus eine Strategie machen will, redet vorher mit einem Steuerberati.

Ob zusätzliches Geld in Sondertilgung oder in andere Ertragsbringer gehört, ist eine Rechenfrage aus Kreditzins, Steuern, Rendite, Risiko und benötigter Liquidität. Das kann für eine vermietete Wohnung anders aussehen als für das eigene Haus. Liquidität hat dabei einen eigenen Wert: Eine bereits getilgte Rate bezahlt keine kaputte Heizung.

Die Lage bleibt, der Rest muß stimmen

Immobilien sind arschig. Es ist aber am Ende Dein Arsch, der frierend und verschuldet darin sitzt.

Also: Tausende Anzeigen wegfiltern. Energiebedarf und reale Verbräuche besorgen. Strom, PV, Wärmepumpe, Ladeanschluß und Glasfaser prüfen. Bei Wohnungen die Eigentümergemeinschaft durchleuchten. Freunde zum Dissen mitnehmen, dann den Gutachter bezahlen. Grundbuch, Pläne und Genehmigungen lesen. Sanierung, Reserve und spätere Einzugskosten vom Kaufpreis abziehen. Finanzierung so bauen, daß ein schlechtes Jahr nicht das Haus frißt.

Und noch einmal: Das ist die Realität, nicht die Sehnsucht fossiler Lobbyisten. Haus, Verkehr und Arbeit hängen an Infrastrukturentscheidungen, Bauvorschriften und Energiepolitik. Wahlen bestimmen diese Zukunft, aber politisches Engagement endet nicht bei der Bundestagswahl. Im Gemeinderat, im Land und in Parteien entscheiden erstaunlich wenige aktive Menschen erstaunlich viel.

Lage, Lage, Lage sagt, wo das Haus steht. Ob es 2026 noch etwas taugt, steht auf einer längeren Liste.

Top

data04: DEGRADED

Post by Dan Langille via Dan Langille's Other Diary »

This morning I noticed this in the logs after doing some pkg upgrade. I was mainly updating openvpn, but in that operation, fail2ban was removed (because I went from python312 to python314). I noticed it missing on one host:

Can't exec "/usr/local/bin/fail2ban-client": No such file or directory at /usr/local/etc/snmp/fail2ban line 116.

I ran this grep to verify fail2ban had been removed from another host:

[12:31 r730-01 dvl ~] % grep fail /var/log/messages
Jun 30 15:38:58 r730-01 upsmon[2715]: Poll UPS [ups04@gw01.int.unixathome.org] failed - Driver not connected
Jun 30 15:39:03 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Connection refused
Jun 30 15:39:28 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 15:40:18 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 15:40:43 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 15:40:53 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Connection refused
Jun 30 21:07:06 r730-01 upsmon[2715]: Poll UPS [ups04@gw01.int.unixathome.org] failed - Driver not connected
Jun 30 21:07:06 r730-01 kernel: Jun 30 21:07:06 r730-01 upsmon[2715]: Poll UPS [ups04@gw01.int.unixathome.org] failed - Driver not connected
Jun 30 21:07:23 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 21:07:23 r730-01 kernel: Jun 30 21:07:23 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 21:07:50 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 21:08:44 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Operation timed out
Jun 30 21:08:51 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Connection refused
Jun 30 21:08:58 r730-01 upsmon[2715]: UPS [ups04@gw01.int.unixathome.org]: connect failed: Connection failure: Connection refused
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing queued i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o

Well, this host doesn’t run fail2ban, but those messages are “interesting”.

How much is nvme mentioned in the logs?

[12:31 r730-01 dvl ~] % grep nvme3 /var/log/messages
Jun 29 18:02:51 r730-01 kernel: nvme3:  mem 0x92000000-0x92003fff at device 0.0 numa-domain 0 on pci7
Jun 29 18:02:51 r730-01 kernel: nda3 at nvme3 bus 0 scbus19 target 0 lun 1
Jun 30 21:11:44 r730-01 kernel: nvme3:  mem 0x92000000-0x92003fff at device 0.0 numa-domain 0 on pci7
Jun 30 21:11:44 r730-01 kernel: nda3 at nvme3 bus 0 scbus19 target 0 lun 1
Jul  2 04:21:30 r730-01 kernel: nvme3: Resetting controller due to a timeout.
Jul  2 04:21:30 r730-01 kernel: nvme3: event="start"
Jul  2 04:21:30 r730-01 kernel: nvme3: Waiting for reset to complete
Jul  2 04:21:50 r730-01 kernel: nvme3: Waiting for reset to complete
Jul  2 04:21:50 r730-01 kernel: nvme3: controller ready did not become 0 within 20500 ms
Jul  2 04:21:50 r730-01 kernel: nvme3: event="timed_out"
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:1 cid:126 nsid:1 lba:5816680880 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:1 cid:126 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:1 cid:125 nsid:1 lba:5819691576 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:1 cid:125 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=5ab381b0 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:1 cid:119 nsid:1 lba:7155944008 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 5, Retries exhausted
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=5ae17238 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:1 cid:119 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 5, Retries exhausted
Jul  2 04:21:50 r730-01 kernel: nvme3: failing queued i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:2 cid:0 nsid:1 lba:3948824896 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:2 cid:0 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=aa870a48 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 5, Retries exhausted
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=eb5e4940 0 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 5, Retries exhausted
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:2 cid:121 nsid:1 lba:5969482320 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:2 cid:121 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nda3 at nvme3 bus 0 scbus19 target 0 lun 1
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:3 cid:125 nsid:1 lba:2128907408 len:1360
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:3 cid:125 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: WRITE (01) sqid:3 cid:116 nsid:1 lba:4236260504 len:16
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:3 cid:116 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=63cf1250 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:3 cid:121 nsid:1 lba:3846619816 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee48c90 0 54f 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): WRITE (01). NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=fc803498 0 f 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:3 cid:121 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=e546c2a8 0 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:4 cid:112 nsid:1 lba:923353176 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:4 cid:112 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:4 cid:115 nsid:1 lba:7403218720 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=37094058 0 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:4 cid:115 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=b9442720 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:6 cid:123 nsid:1 lba:4092436328 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:6 cid:123 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:6 cid:117 nsid:1 lba:4119853392 len:8
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f3ed9f68 0 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:6 cid:117 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f58ff950 0 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: FLUSH (00) sqid:11 cid:123 nsid:1
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:11 cid:123 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): FLUSH (00). NCB: opc=0 fuse=0 nsid=1 prp1=0 prp2=0 cdw=0 0 0 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:119 nsid:1 lba:2128908768 len:2048
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:119 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:121 nsid:1 lba:2128910816 len:2048
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:121 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:123 nsid:1 lba:2128912864 len:1368
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:123 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:122 nsid:1 lba:2128914232 len:2048
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee491e0 0 7ff 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:122 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee499e0 0 7ff 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee4a1e0 0 557 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:126 nsid:1 lba:2128916280 len:2048
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:126 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee4a738 0 7ff 0 0 0
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:14 cid:125 nsid:1 lba:2128918328 len:1368
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:125 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:15 cid:121 nsid:1 lba:7603680584 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:15 cid:121 cdw0:0
Jul  2 04:21:50 r730-01 kernel: nvme3: failing outstanding i/o
Jul  2 04:21:50 r730-01 kernel: nvme3: READ (02) sqid:16 cid:126 nsid:1 lba:4406184968 len:8
Jul  2 04:21:50 r730-01 kernel: nvme3: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:16 cid:126 cdw0:0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee4af38 0 7ff 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=7ee4b738 0 557 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=c536f548 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=6a10c08 1 7 0 0 0
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): CAM status: NVME Status Error
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Error 6, Periph was invalidated
Jul  2 04:21:50 r730-01 kernel: nvme3: Failed controller, stopping watchdog timeout.
Jul  2 04:21:50 r730-01 kernel: (nda3:nvme3:0:0:1): Periph destroyed
Jul  2 04:21:50 r730-01 kernel: nvme3: Failed controller, stopping watchdog timeout.

Well, that’s a lot. How’s the status?

[12:31 r730-01 dvl ~] % zpool status
  pool: data01
 state: ONLINE
  scan: scrub repaired 0B in 00:00:07 with 0 errors on Thu Jul  2 03:48:55 2026
config:

	NAME                  STATE     READ WRITE CKSUM
	data01                ONLINE       0     0     0
	  raidz2-0            ONLINE       0     0     0
	    gpt/Y7P0A022TEVE  ONLINE       0     0     0
	    gpt/Y7P0A02ATEVE  ONLINE       0     0     0
	    gpt/Y7P0A02DTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02GTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02LTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02MTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02QTEVE  ONLINE       0     0     0
	    gpt/Y7P0A033TEVE  ONLINE       0     0     0

errors: No known data errors

  pool: data02
 state: ONLINE
  scan: scrub repaired 0B in 00:03:59 with 0 errors on Thu Jul  2 03:52:59 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data02                   ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    gpt/S6WSNJ0T208743F  ONLINE       0     0     0
	    gpt/S6WSNJ0T207774T  ONLINE       0     0     0

errors: No known data errors

  pool: data03
 state: ONLINE
  scan: scrub repaired 0B in 01:16:19 with 0 errors on Thu Jul  2 05:05:31 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data03                   ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    gpt/WD_22492H800867  ONLINE       0     0     0
	    gpt/WD_230151801284  ONLINE       0     0     0
	  mirror-1               ONLINE       0     0     0
	    gpt/WD_230151801478  ONLINE       0     0     0
	    gpt/WD_230151800473  ONLINE       0     0     0

errors: No known data errors

  pool: data04
 state: DEGRADED
status: One or more devices have been removed.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using zpool online' or replace the device with
	'zpool replace'.
  scan: scrub repaired 0B in 01:11:17 with 0 errors on Thu Jul  2 05:00:37 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   DEGRADED     0     0     0
	  raidz2-0               DEGRADED     0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  REMOVED      0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
	The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:53 with 0 errors on Thu Jul  2 03:50:16 2026
config:

	NAME                               STATE     READ WRITE CKSUM
	zroot                              ONLINE       0     0     0
	  mirror-0                         ONLINE       0     0     0
	    gpt/zfs0_20170718AA0000185556  ONLINE       0     0     0
	    gpt/zfs1_20170719AA1178164201  ONLINE       0     0     0

errors: No known data errors

Then I checked Nagios – it had found the same issue. I hadn’t check Nagios before today. Oh oh.

Let’s look at that device:

[12:47 r730-01 dvl ~] % sudo nvmecontrol identify nvme3
nvmecontrol: Identify request failed
[12:47 r730-01 dvl ~] % 

I went to LibreNMS to see if there was any trending information about that drive. It was not found. I suspect when it dropped out, LibreNMS also dropped it. If that’s the case, that’s not helpful.

Let’s try a reboot.

After a reboot, that drive (S7KGNU0Y912955D) was not found. My next idea: open up the case and reseat that device.

I’m hoping that device is not dead. It went into service 7 months ago and prices have jumped more than slightly lately.

When I checked another device:

[13:16 r730-01 dvl ~] % sudo smartctl -a /dev/nvme4
smartctl 7.5 2025-04-30 r5714 [FreeBSD 15.0-RELEASE-p11 amd64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 990 EVO Plus 4TB
...
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    1%
Data Units Read:                    76,836,586 [39.3 TB]
Data Units Written:                 47,333,046 [24.2 TB]
Host Read Commands:                 2,617,490,888
Host Write Commands:                1,200,934,619
Controller Busy Time:               8,225
Power Cycles:                       23
Power On Hours:                     6,562
...

That usage level is not outrageous. All units in this zpool should be more-or-less identically used.

Drive is not dead

I powered off the host, and pulled out the ASUS Hyper M.2 X16 Gen 4 card. I move the NVMe card in question to a portable carrier. I hooked that up to my Macbook. It was identified as a “Samsung SSD 990 PRO 4TB” – that tells me it’s not completely dead.

bsdimp suggested I hook that up to a FreeBSD box.

While monitoring /var/log/messages, I did just not. Nothing. :(

I tried another USB port; nothing. I then tried a USB port on the back of the host:

Jul  2 15:02:59 r730-03 kernel: da8 at umass-sim1 bus 1 scbus19 target 0 lun 0
Jul  2 15:02:59 r730-03 kernel: da8: <Samsung SSD 990 PRO 4TB 1.00> Fixed Direct Access SPC-4 SCSI device
Jul  2 15:02:59 r730-03 kernel: da8: Serial Number 01293805127E
Jul  2 15:02:59 r730-03 kernel: da8: 40.000MB/s transfers
Jul  2 15:02:59 r730-03 kernel: da8: 3815447MB (7814037168 512 byte sectors)
Jul  2 15:02:59 r730-03 kernel: da8: quirks=0x2<NO_6_BYTE>

That give me hope. As does this:

[15:03 r730-03 dvl ~] % sudo smartctl -a /dev/da8
smartctl 7.5 2025-04-30 r5714 [FreeBSD 15.0-RELEASE-p11 amd64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 990 PRO 4TB
Serial Number:                      S7KGNU0Y912955D
Firmware Version:                   4B2QJXD7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 4,000,787,030,016 [4.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       2.0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          4,000,787,030,016 [4.00 TB]
Namespace 1 Utilization:            3,057,326,026,752 [3.05 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 4951a0eec6
Local Time is:                      Thu Jul  2 15:03:56 2026 UTC
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0055):     Comp DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x2f):         S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Log0_FISE_MI
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     82 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.39W       -        -    0  0  0  0        0       0
 1 +     9.39W       -        -    1  1  1  1        0       0
 2 +     9.39W       -        -    2  2  2  2        0       0
 3 -   0.0400W       -        -    3  3  3  3     4200    2700
 4 -   0.0050W       -        -    4  4  4  4      500   21800

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        34 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    56,743,286 [29.0 TB]
Data Units Written:                 11,391,387 [5.83 TB]
Host Read Commands:                 2,383,774,544
Host Write Commands:                425,045,375
Controller Busy Time:               802
Power Cycles:                       21
Power On Hours:                     5,614
Unsafe Shutdowns:                   10
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               34 Celsius
Temperature Sensor 2:               36 Celsius

Warning: NVMe Get Log truncated to 0x200 bytes, 0x200 bytes zero filled
Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

Warning: NVMe Get Log truncated to 0x200 bytes, 0x034 bytes zero filled
Self-test Log (NVMe Log 0x06, NSID 0xffffffff)
Self-test status: No self-test in progress
No Self-tests Logged

Back into the box

I disconnected that mobile carrier from the FreeBSD USB port. I installed it back onto the PCIe card, swapping it with another device. It was in the slot farthest from the fan. Now it’s one slot closer to the fan.

I booted up the host. And I see:

[15:29 r730-01 dvl ~] % zpool status data04
  pool: data04
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Jul  2 15:28:49 2026
	1.73T / 9.30T scanned, 10.9G / 7.58T issued at 1.81G/s
	1.84G resilvered, 0.14% done, 01:11:23 to go
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     2  (resilvering)
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

This is as good as can be expected. :)

About 10 minutes later:

[15:29 r730-01 dvl ~] % zpool status data04
  pool: data04
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: resilvered 1.84G in 00:00:20 with 0 errors on Thu Jul  2 15:29:09 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     2
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

All good. Let’s do a scrub before I clear out those errors.

[15:39 r730-01 dvl ~] % sudo zpool scrub data04
[15:40 r730-01 dvl ~] % zpool status data04    
  pool: data04
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: scrub in progress since Thu Jul  2 15:40:05 2026
	182G / 9.30T scanned at 45.6G/s, 0B / 9.30T issued
	0B repaired, 0.00% done, no estimated completion time
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     2
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

Logs

Let’s find that device.

[15:41 r730-01 dvl ~] % grep S7KGNU0Y912955D /var/run/dmesg.boot 
nda2: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y912955D>
nda2: Serial Number S7KGNU0Y912955D

It is now nda2 – given it was nvme3 before, and it moved slots, I figure that’s our device.

[15:43 r730-01 dvl ~] % sudo nvmecontrol logpage -p 2 nvme2
SMART/Health Information Log
============================
Critical Warning State:         0x00
 Available spare:               0
 Temperature:                   0
 Device reliability:            0
 Read only:                     0
 Volatile memory backup:        0
Temperature:                    313 K, 39.85 C, 103.73 F
Available spare:                100
Available spare threshold:      10
Percentage used:                0
Data units (512,000 byte) read: 56780042
Data units written:             11396179
Host read commands:             2385648136
Host write commands:            425345148
Controller busy time (minutes): 802
Power cycles:                   22
Power on hours:                 5614
Unsafe shutdowns:               10
Media errors:                   0
No. error info log entries:     0
Warning Temp Composite Time:    0
Error Temp Composite Time:      0
Temperature Sensor 1:           313 K, 39.85 C, 103.73 F
Temperature Sensor 2:           326 K, 52.85 C, 127.13 F
Temperature 1 Transition Count: 0
Temperature 2 Transition Count: 0
Total Time For Temperature 1:   0
Total Time For Temperature 2:   0

An hour or so later

Phew.


[16:50 r730-01 dvl ~/bin] % zpool status data04
  pool: data04
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: scrub repaired 0B in 01:10:47 with 0 errors on Thu Jul  2 16:50:52 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     2
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors
[16:51 r730-01 dvl ~/bin] % sudo zpool clear data04
[16:51 r730-01 dvl ~/bin] % zpool status data04    
  pool: data04
 state: ONLINE
  scan: scrub repaired 0B in 01:10:47 with 0 errors on Thu Jul  2 16:50:52 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors
[16:51 r730-01 dvl ~/bin] % 

Seems OK now.

The next day

The next day, Friday Jul 3 2026, all was still well. I went looking for temperatures spikes.

I found on, only for the basement room temperature. It had hit 80F for the first time since last summer. See graphs below.

It could be that was enough to loosen the connection between that M.2 device and its connector. This device, and others, were installed in December of 2025 (about 5 months ago) when the temperature was about 55F. NOTE: This stick was the farthest from the fan. If the next device to drop is S7KGNU0Y912937J, we’ll know the position is relevant.

7 day trend
7 day trend
the past year
the past year
Top

data04: DEGRADED – oh and have I mentioned WatchGoose?

Post by Dan Langille via Dan Langille's Other Diary »

I saw this today in my logs on r730-01. It looks similar to a previous DEGRADED even back on July 2. In this case, the device is S7KGNU0Y915666E, not S7KGNU0Y912955D as it was back then.

Sep 12 03:46:54 r730-01 kernel: nvme1: Resetting controller due to a timeout.
Sep 12 03:46:54 r730-01 kernel: nvme1: event="start"
Sep 12 03:46:54 r730-01 kernel: nvme1: Waiting for reset to complete
Sep 12 03:47:14 r730-01 syslogd: last message repeated 197 times
Sep 12 03:47:14 r730-01 ZFS[96518]: vdev I/O failure, zpool=data04 path=/dev/gpt/S7KGNU0Y915666E offset=270336 size=8192 error=6
Sep 12 03:47:14 r730-01 kernel: nvme1: Waiting for reset to complete
Sep 12 03:47:14 r730-01 kernel: nvme1: controller ready did not become 0 within 20500 ms
Sep 12 03:47:14 r730-01 kernel: nvme1: event="timed_out"
Sep 12 03:47:14 r730-01 kernel: nvme1: failing outstanding i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: FLUSH (00) sqid:1 cid:112 nsid:1
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:1 cid:112 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: failing outstanding i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:120 nsid:1 lba:2320185448 len:2048
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): FLUSH (00). NCB: opc=0 fuse=0 nsid=1 prp1=0 prp2=0 cdw=0 0 0 0 0 0
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:120 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:121 nsid:1 lba:2320187496 len:2048
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:121 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:123 nsid:1 lba:2320189544 len:1368
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:123 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b3868 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:118 nsid:1 lba:2320190912 len:2048
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b4068 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b4868 0 557 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:118 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:125 nsid:1 lba:2320192960 len:2048
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b4dc0 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:125 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:113 nsid:1 lba:2320195008 len:1368
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b55c0 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:113 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 5, Retries exhausted
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:115 nsid:1 lba:2320196376 len:2048
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:115 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:114 nsid:1 lba:2320198424 len:2048
Sep 12 03:47:14 r730-01 kernel: nda1 at nvme1 bus 0 scbus17 target 0 lun 1
Sep 12 03:47:14 r730-01 kernel: nda1: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y915666E> s/n S7KGNU0Y915666E detached
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:114 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:5 cid:122 nsid:1 lba:2320200472 len:1368
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:5 cid:122 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: failing queued i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:7 cid:0 nsid:1 lba:4317358168 len:8
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b5dc0 0 557 0 0 0
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:7 cid:0 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:7 cid:0 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: nvme1: failing outstanding i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: WRITE (01) sqid:11 cid:125 nsid:1 lba:4127749432 len:8
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:11 cid:125 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: WRITE (01) sqid:11 cid:119 nsid:1 lba:5324663584 len:8
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:11 cid:119 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: failing queued i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:13 cid:0 nsid:1 lba:5554129904 len:8
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:13 cid:0 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: failing outstanding i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:14 cid:115 nsid:1 lba:4123885016 len:32
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:115 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: READ (02) sqid:14 cid:125 nsid:1 lba:4123885456 len:8
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:14 cid:125 cdw0:0
Sep 12 03:47:14 r730-01 kernel: nvme1: failing outstanding i/o
Sep 12 03:47:14 r730-01 kernel: nvme1: WRITE (01) sqid:15 cid:104 nsid:1 lba:4127749424 len:8
Sep 12 03:47:14 r730-01 kernel: nvme1: ABORTED_BY_REQUEST (00/07) crd:0 m:0 dnr:1 p:0 sqid:15 cid:104 cdw0:0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b6318 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b6b18 0 7ff 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=8a4b7318 0 557 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=155a858 1 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): WRITE (01). NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f6087538 0 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): WRITE (01). NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=3d5feb20 1 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=4b0d4bf0 1 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f5cd7dd8 0 1f 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): READ (02). NCB: opc=2 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f5cd7f90 0 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): WRITE (01). NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=f6087530 0 7 0 0 0
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): CAM status: NVME Status Error
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): NVMe status: ABORTED_BY_REQUEST (00/07) DNR
Sep 12 03:47:14 r730-01 kernel: (nda1:nvme1:0:0:1): Error 6, Periph was invalidated
Sep 12 03:47:14 r730-01 kernel: nvme1: Failed controller, stopping watchdog timeout.
Sep 12 03:47:15 r730-01 ZFS[96546]: vdev state changed, pool_guid=1760090623298529734 vdev_guid=10692465753147695115
Sep 12 03:47:15 r730-01 ZFS[96550]: vdev is removed, pool_guid=1760090623298529734 vdev_guid=10692465753147695115
Sep 12 03:47:15 r730-01 kernel: nvme1: Failed controller, stopping watchdog timeout.
Sep 12 03:47:15 r730-01 syslogd: last message repeated 1 times
Sep 12 03:47:15 r730-01 syslogd: last message repeated 1 times
Sep 12 03:47:15 r730-01 kernel: (nda1:nvme1:0:0:1): Periph destroyed
Sep 12 03:47:15 r730-01 kernel: nvme1: Failed controller, stopping watchdog timeout.
Sep 12 03:47:15 r730-01 syslogd: last message repeated 1 times

The zpool status is:

[12:16 r730-01 dvl ~] % zpool status data04
  pool: data04
 state: DEGRADED
status: One or more devices have been removed.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using zpool online' or replace the device with
	'zpool replace'.
  scan: scrub repaired 0B in 01:20:36 with 0 errors on Sat Sep 12 04:29:12 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   DEGRADED     0     0     0
	  raidz2-0               DEGRADED     0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  REMOVED      0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

What’s next?

I will reboot this host after dbclone finishes restoring database backups

.

I’ll take this opportunity to mention WatchGoose – a third party website for monitoring tasks, such as cronjobs. If I’d checked here first, I’d have seen dbclone had not finished yet today. There is a free tier for this service.

WatchGoose
WatchGoose

Full disclosure: They offered free accounts to FreeBSD Developers. I got one. And I plan to add more monitoring such as:

  • has the Bacula database backup completed? (one host)
  • has the Bacula database backup been rsynced? (three hosts)
  • has the Bacula configuration been rsynced? (three hosts)
  • have the zfs snapshots been created? (several hosts)
  • have the data backup rsyncs run? (several hosts)
  • have the FreshPorts package checks been pulled? (several hosts)
  • have the database backups been rsynced to dbclone? (several hosts)

It’s mainly a matter of time for implementing that.

I plan to post this, then create another post as I resolve the issue.

After the reboot

After the reboot, the missing device was not seen in /var/run/dmesg.boot.

I powered off the host, opened it up, and pulled the NVMe card, and removed the heatsink.

The 4x NVMe sticks
The 4x NVMe sticks

The stick in question is the 2nd slot from the fan. In the previous post, the problem stick was described as: “It was in the slot farthest from the fan. Now it’s one slot closer to the fan”.

That tells me this incident is not related to the stick or to the slot on the card.

This time, I removed the screw holding down the NVMe stick, removed the stick, reseated it, and reinstalled the screw.

I reinstalled the card, closed the server, hooked up all the cables, and powered on.

My plan: go into single user modem run service jail disable and proceed into multi user mode.

That didn’t stop the jails from starting up….

However, the zpool status is fine:

[16:18 r730-01 dvl ~] % zpool status data04
  pool: data04
 state: ONLINE
  scan: resilvered 3.03G in 00:00:31 with 0 errors on Sat Sep 12 16:18:20 2026
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors
[16:18 r730-01 dvl ~] % 

Why did the jails start?

Based on this, my command didn’t take, and I don’t know why.

[16:19 r730-01 dvl ~] % grep jail /etc/rc.conf
jail_enable="YES"
jail_reverse_stop="YES"
jail_list="logs dns1 cliff2 mysql01 mysql02 pg01 pg02 pg03 pkg01 dev-ingress01             dvl-ingress01             test-ingress01              stage-ingress01                bacula besser certs-rsync certs git svn webserver mqtt01 bacula-sd-02 talos samdrucker bacula-sd-03 mydev jail_within_jail serpico dns-hidden-master nsnotify unifi01 dev-nginx01 dvl-nginx01 test-nginx01 stage-nginx01"

The file has not been touched in months:

[16:22 r730-01 dvl /etc/rc.conf.d] % cd
[16:22 r730-01 dvl ~] % date
Sat Sep 12 16:22:04 UTC 2026
[16:22 r730-01 dvl ~] % ls -l /etc/rc.conf
-rw-r--r--  1 root wheel 3410 2026.06.29 18:06 /etc/rc.conf

User error? I tried scrolling back to confirm my command, but it has scrolled off my console terminal.

Top

Git Worktrees: mehrere Branches gleichzeitig auschecken

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

Ein Git-Repository kann mehrere Arbeitsverzeichnisse haben. Mit git worktree liegt main zum Beispiel in ~/Source/project, während der Branch new-feature gleichzeitig in ~/Source/project/.worktrees/new-feature ausgecheckt ist.

Das ist nützlich, wenn im Hauptverzeichnis ein Server läuft, ein Test lange arbeitet oder man eine angefangene Änderung nicht wegräumen will, nur um kurz einen anderen Branch anzusehen. Ein zweiter Clone würde ebenfalls funktionieren, dupliziert aber die Repository-Daten und muß separat aktuell gehalten werden. Worktrees teilen sich dagegen dasselbe Repository.

Worktrees und Branches

Ein Branch ist in Git zunächst nur ein beweglicher Name für einen Commit. Ein Worktree ist ein Arbeitsverzeichnis mit ausgecheckten Dateien, einem eigenen HEAD und einem eigenen Index, also einer eigenen Staging Area.

Im Normalfall ist jeder Worktree mit genau einem Branch verbunden. Git läßt denselben Branch nicht gleichzeitig in zwei Worktrees auschecken. Sonst könnten zwei Arbeitsverzeichnisse unabhängig voneinander versuchen, denselben Branch-Zeiger zu verschieben.

Für die Beispiele sieht die Verzeichnisstruktur so aus:

~/Source/project/ main
└── .worktrees/
 ├── new-feature/ new-feature
 └── linear-feature/ linear-feature

Das Projekt enthält diesen Eintrag in .gitignore:

/.worktrees

Der führende Slash beschränkt die Regel auf .worktrees im Wurzelverzeichnis des Projektes. Git ignoriert dadurch die Arbeitsverzeichnisse der anderen Worktrees im Checkout von main.

Einen Feature-Worktree anlegen

Wir beginnen im bestehenden Checkout. main zeigt auf den Commit, von dem das Feature abzweigen soll:

cd ~/Source/project
git switch main
git status --short
git worktree add -b new-feature .worktrees/new-feature

-b new-feature legt den Branch an. Das letzte Argument bestimmt das Arbeitsverzeichnis. Ohne weiteren Commit oder Branch als Startpunkt verwendet Git dabei HEAD, hier also den aktuellen Stand von main.

Das Verzeichnis ist danach ein vollständiges Arbeitsverzeichnis:

cd ~/Source/project/.worktrees/new-feature
git branch --show-current

Die Ausgabe ist new-feature. Änderungen in diesem Verzeichnis gehören zu diesem Worktree. Das gilt auch für den Index: git add im Feature-Worktree verändert nicht die Staging Area im Hauptverzeichnis.

Implementieren, testen und committen

Nun wird das Feature wie in jedem anderen Checkout bearbeitet. Die konkreten Dateien und der Testbefehl hängen vom Projekt ab:

cd ~/Source/project/.worktrees/new-feature
$EDITOR src/feature.py tests/test_feature.py
pytest -v
git status --short
git add src/feature.py tests/test_feature.py
git commit -m "Implement new feature"

Der Commit verschiebt den Branch new-feature. main bleibt unverändert. Die beiden Branches zeigen jetzt auf unterschiedliche Commits, obwohl ihre Worktrees dasselbe Git-Repository benutzen.

In main mergen und aufräumen

Wir wechseln nicht mit git switch zwischen den Branches, sondern gehen einfach in den bereits vorhandenen Worktree von main:

cd ~/Source/project
git switch main
git merge --no-ff new-feature
pytest -v

--no-ff erzeugt hier absichtlich einen Merge-Commit. Das Feature bleibt dadurch als eigener Zweig in der Historie sichtbar, auch wenn ein Fast-Forward möglich gewesen wäre.

Erst wenn der Merge und die Tests sicher sind, räumen wir auf:

cd ~/Source/project
git worktree remove .worktrees/new-feature
git branch -d new-feature

Die Reihenfolge ist wichtig. Solange der Branch im Worktree ausgecheckt ist, läßt Git ihn nicht löschen. git worktree remove verweigert außerdem ohne --force das Entfernen eines Worktrees mit nicht committeten Änderungen, und git branch -d verweigert das Löschen eines nicht gemergten Branches. Diese Sicherungen sollte man nicht ohne konkreten Grund abschalten.

Lineare Historie mit Rebase und Fast-Forward

Im zweiten Beispiel soll kein Merge-Commit entstehen. Wir legen wieder einen Worktree samt Branch an und entwickeln darin:

cd ~/Source/project
git worktree add -b linear-feature .worktrees/linear-feature

cd .worktrees/linear-feature
$EDITOR src/linear_feature.py tests/test_linear_feature.py
pytest -v
git add src/linear_feature.py tests/test_linear_feature.py
git commit -m "Implement linear feature"

Währenddessen kann main durch andere Arbeit weitergelaufen sein. Vor dem Merge setzen wir die Feature-Commits deshalb auf den aktuellen Stand von main:

cd ~/Source/project/.worktrees/linear-feature
git rebase main
pytest -v

Rebase nimmt die Commits, die nur auf linear-feature existieren, und spielt sie auf dem aktuellen Commit von main neu ab. Dabei entstehen neue Commit-IDs. Das ist für einen lokalen, noch nicht veröffentlichten Feature-Branch unproblematisch.

Falls Konflikte auftreten, werden sie im Feature-Worktree aufgelöst. Danach geht es mit git rebase --continue weiter. Nach erfolgreichem Rebase kann main ohne Merge-Commit vorwärtsgeschoben werden:

cd ~/Source/project
git merge --ff-only linear-feature
pytest -v
git worktree remove .worktrees/linear-feature
git branch -d linear-feature

--ff-only ist hier die entscheidende Kontrolle. Ist main seit dem Rebase erneut weitergelaufen, bricht Git ab, statt doch einen Merge-Commit zu bauen. Dann wird der Feature-Branch noch einmal auf main rebased.

Worktrees verwalten

Kommando Zweck
git worktree list Alle Worktrees mit Pfad, Commit und Branch anzeigen
git worktree add -b NAME PFAD Neuen Branch und Worktree von HEAD anlegen
git worktree add PFAD BRANCH Bestehenden Branch in einem neuen Worktree auschecken
git worktree remove PFAD Einen sauberen Worktree entfernen
git worktree move PFAD NEUER-PFAD Einen Worktree verschieben
git worktree lock PFAD Automatisches Aufräumen eines zeitweise nicht erreichbaren Worktrees verhindern
git worktree unlock PFAD Einen gesperrten Worktree wieder freigeben
git worktree prune Verwaiste Verwaltungsdaten gelöschter Worktrees entfernen
git worktree repair Verknüpfungen nach manuellem Verschieben reparieren

Für Skripte liefert git worktree list --porcelain ein stabiler zu verarbeitendes Format. Im Alltag reichen meist add, list und remove.

Wie Worktrees implementiert sind

Logisch kann man sich einen Worktree als weiteren Clone vorstellen, dessen .git auf das Repository im Hauptverzeichnis zeigt. Das Bild erklärt den wichtigsten Effekt: Jeder Worktree hat eigene ausgecheckte Dateien, aber alle sehen dieselben Commits und Branches.

Technisch ist .git im verknüpften Worktree jedoch kein Symlink und kein Verzeichnis, sondern eine Textdatei. Sie verweist auf ein Verwaltungsverzeichnis unter .git/worktrees/ des Haupt-Worktrees, ungefähr so:

gitdir: /home/user/Source/project/.git/worktrees/new-feature

Dieses Verwaltungsverzeichnis enthält unter anderem den eigenen HEAD, den eigenen Index und einen Rückverweis auf den Worktree. Eine commondir-Datei verweist von dort auf das gemeinsame .git-Verzeichnis. Objekt-Datenbank, Branches, Tags und die meisten anderen Repository-Daten werden gemeinsam benutzt; Arbeitsdateien, HEAD und Index sind pro Worktree getrennt.

Darum braucht ein Worktree kaum zusätzlichen Platz jenseits seiner ausgecheckten Dateien. Darum sieht er neue Commits und Branches sofort. Und darum sollte man ihn mit git worktree remove statt durch bloßes Löschen des Verzeichnisses entfernen: Git muß nicht nur Dateien, sondern auch seine Verwaltungsdaten aufräumen.

Top

Microsoft Plugs Nearly 1,000 Security Holes

Post by Brian Krebs via Krebs on Security »

Microsoft Corp. today issued updates to plug at least 974 security holes in its Windows operating systems and other software, by far its biggest single patch batch ever. Microsoft says artificial intelligence is helping to speed the discovery of vulnerabilities, but security experts warn that many organizations already are struggling to prioritize the more human-intensive endeavor of testing and deploying so many fixes each month.

Image: Shutterstock.com, Kirill Makarov.

This month’s patch bundle obliterates the software giant’s previous record set in July, when it released updates for at least 570 security vulnerabilities. September’s Patch Tuesday brings this year’s total to more than 2,600, more than twice Microsoft’s previous record-setting patch year in 2020 (1,245) and with three more months to go.

There are two “zero-day” flaws fixed this month that are being actively exploited: both CVE-2026-81963 and CVE-2026-85880 allow an attacker to elevate their privileges on Windows system.

Fully 113 of the bugs addressed today earned Microsoft’s “critical” rating, meaning they could be abused by malware or miscreants to seize control over a vulnerable Windows machine with little or no help from the user.

Among the more serious critical flaws this month is CVE-2026-69730, a DNS weakness present in Windows Server 2012 onward and on Windows 10. Microsoft warns that an unauthenticated attacker could leverage this weakness simply by sending a specially crafted packet to an affected system, and that it is likely to be exploited.

Also scary is CVE-2026-69829, a critical, remote code execution flaw in the Windows Shell. This vulnerability has a CVSS base score of 9.8 (10 is the most severe), and can be exploited with low attack complexity, no privileges, and no user interaction.

Microsoft’s summary of the security updates released today. Image: msrc.microsoft.com.

Microsoft is hardly alone in shipping monster patch bundles lately. Many other large software companies, including Adobe, Cisco, Google, Mozilla and Oracle, all have recently credited AI-assisted research with increasing their patch cadence and volume (Google said today it is now going to ship security updates every two weeks).

Tyler Reguly, associate director of security research and development at Fortra, said one core challenge with deploying Windows updates is that they need to be tested before being installed across an organization because not all third-party software works seamlessly in the face of changes to the underlying operating system.

“It’s time to put our CISOs and CSOs on notice,” Reguly said. “How are you helping your teams through these difficult times? Do you have your teams deploy after hours and on weekends to avoid disruption to the business environment? Do you reward them for that effort? Time to dig into your budget and buy dinner for your teams that are working on Saturday to get patches rolled out before users return to work on Monday.”

Satnam Narang is senior staff research engineer at Tenable. Narang said it’s important to recognize that while the number of vulnerabilities being patched by Microsoft is rising, the number of flaws that can and will affect most organizations remains quite low.

“AI-assisted vulnerability discovery in 2026 is creating larger haystacks, but it isn’t finding more needles,” he said. “It’s critical that organizations understand which vulnerabilities actually apply to them, whether they pose a threat by being reachable and exploitable, and prioritize remediation based on this risk context.”

Of course, regular Windows users don’t need to test patches before deploying them, but they still need to open Windows Update periodically or else assent to the program’s nag notices about pending updates. And at the rate these Windows patch releases are ballooning in size, it’s probably best not to let them pile up month after month.

Enterprise Windows admins will want to keep an eye on askwoody.com for news of any updates that appear to be causing problems. As always, the SANS Internet Storm Center has a per-patch breakdown ordered by severity and urgency.

Top

FreeBSD 14.5-RELEASE Available

Post by FreeBSD Newsflash via FreeBSD News Flash »

Release Information page.
Top

Valuable News – 2026/09/07

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD Git Weekly: 2026-08-24 to 2026-08-30.
https://freebsd-git-weekly.tarsnap.net/2026-08-24.html

Caching freebsd-update(8) and pkg(8) Files with NGINX.
https://omussell.github.io/fbsd-update-cache/

BaroPAM Installation Guide on FreeBSD.
https://mc529.tistory.com/1567

LibreOffice 26.8 is Out – Local First – No AI.
https://theregister.com/applications/2026/08/28/libreoffice-26.8-out/5293301

Call for Testers with NVMe on FreeBSD.
https://lists.freebsd.org/archives/freebsd-current/2026-September/010763.html

Offshoots of Cancelled TrueNAS CORE Upgrade to FreeBSD 15.
https://theregister.com/storage/2026/09/02/offshoots-of-cancelled-truenas-core-upgrade-to-freebsd-15/5293725

Report periodic(8) Sends Anyway.
https://vivianvoss.net/blog/periodic-sheet

FreeBSD Release Cycle: What It Means for Your Upgrade Schedule.
https://klarasystems.com/articles/freebsd-release-cycle-and-upgrade-schedule/

Wine 11.17 Released with Initial Support for Display Mode Emulation.
https://phoronix.com/news/Wine-11.17-Released

AppJail 5.5.0 Released.
https://github.com/DtxdF/AppJail/releases/tag/v5.5.0

NetBSD 9.5 Released.
https://netbsd.org/releases/formal-9/NetBSD-9.5.html

NetBSD 11 from Scratch.
https://meanmicio.org/2026/09/06/netbsd-11-from-scratch/

AMD Based FreeBSD Desktop Reloaded.
https://vermaden.wordpress.com/2026/09/06/amd-based-freebsd-desktop-reloaded/

FreeBSD Introduces Similar to top(1) Tool nfsdtop(8) with NFS Server I/O Using dtrace(1).
https://cgit.freebsd.org/src/commit/?id=60c7313074b93adf4ea70ff44b78b3b4fc15c29f

FreeBSD Git Weekly: 2026-08-31 to 2026-09-06.
https://freebsd-git-weekly.tarsnap.net/2026-08-31.html

Moving from Binary Packages to pkgsrc.
https://eugene-andrienko.com/2026-09-07-bsd-pkgsrc-ports.html

UNIX/Audio/Video

FreeBSD Basics – Part 1 – System /bin/sh Shell.
https://youtube.com/watch?v=sbvzOGv8lm8

BSD Now 679: It Also Goes to 11.
https://www.bsdnow.tv/679

Hardware

ThinkPad Endgame.
https://williamjansson.com/posts/thinkpad_endgame/

How EV Tires are Made Differently from Regular Tires?
https://youtube.com/watch?v=lfjc4XyU0Og

Life

What the Fuck Happened to Nerds.
https://mrmarket.lol/what-the-fuck-happened-to-nerds/

Can Tech Legends Find the Liar – Mafia Episode 1.
https://youtube.com/watch?v=EDCwQe7P8T0

Other

Balkanization of Virtualization will Dethrone VMware.
https://theregister.com/virtualization/2026/08/31/balkanization-virtualization-vmware/5292488

82MHz – Linkdump No 123.
https://82mhz.net/posts/2026/09/linkdump-no-123/

Porsche Unleashed.
https://www.filfre.net/2026/09/porsche-unleashed/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

AMD Based FreeBSD Desktop Reloaded

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

My previous AMD Based FreeBSD Desktop is still up and running – it will now be my kids computer … so I needed a new one … but not a bigger one 🙂 I spent way too much time finding new Mini-ITX case for the next one … and after all possibilities … I got Silverstone case again :] … because nothing else was this small and with all needed features. Also … I was thinking WHAT exactly should I buy – especially in the times with AI prices. For one of my older projects that failed (I hope to also publish about that some day) I had AMD Ryzen 4750GE CPU with 35W TDP but still 8C/16T CPU on AMD4 socket … so instead of killing all price/performance efficiency with DDR5 and AM5 setup … I went to decent AM4 option.

Case

Older build used Silverstone SG05 case and this one uses Silverstone SG13 … which is almost the same.

  WHAT     SG05     SG13
 Width    218mm    222mm
Height    175mm    181mm
 Depth    276mm    285mm
Volume    10.5L    10.8L

So one can say that its just a tiny 5% increase in size … but a welcome and needed one.

Difference

The new setup is about 25% faster when it comes to CPU and 40-60% faster in GPU department … depending on the workload. The SSD/NVMe speeds will be the same as its still the same AM4 platform … and RAM would also be similar as its again DDR4 sticks.

Assembly

After having the parts laying around for about a quarter (yes I was busy) my son finally forced me to make that build … and it was a positive push – we sit down in dining room with all needed parts and I started to assembly it with my son being more then curious how it is to actually build a computer.

First CPU into the motherboard … and NVME into the M.2 slot.

Next thermal paste and fan onto the CPU.

To be honest it was easier said then done as the Thermalight AXP-90 X47 comes with install options for both AMD and Intel solutions and it took me a little time to check all needed screws to use and their accessories.

Next – motherboard into the case.

Next RAM or GPU … as GPU was measured to fit within millimeters in that case I started with the GPU.

… and no matter how I tried … no matter which angles I tried to use … GPU would just not fit into the case 😦

I found funny that the GPU name was Challenger and it was REALLY challenging to fit it into the case.

I even decided to name that system challenger after that. I am a big fan of Dodge Challenger cars … I own a Dodge personally … just not a Challenger … yet 🙂

I inspected the GPU from every possible direction and one detail caught my attention …

The plastic cover with the fans was about 1cm wider then the PCB for the card … so I decided to cut it precisely at that place.

This is how the plastic cover looked like after being cut to the PCB length.

Here is how it went with the cutting – something like 2-3 minutes of work.

… and after reattaching the cover to the GPU card.

I went back to trying to fit the GPU into the case … but the cut was not enough … it would still not fit … so I changed strategy. I dissembled the expansion slot bracket from the GPU – out that expansion slot bracket into its place in the case – and tried to fit the card again – so later I can add screws to expansion slot bracket to attach it to GPU again … but I was not able to fit all those again. So next ‘victim’ of cutting was the expansion slot bracket … before on the picture below.

… and after 🙂

Not much was needed – but without that cut – fitting that Challenger into the case was not possible.

After that I was able to finally attach the GPU to the motherboard.

It was more or less like that.

… and after the additional bracket cover.

Next for the installation were RAM sticks and SSD SATA drive – nothing spectacular about that – after also adding the PSU it was more or less complete.

Its really nice that Thermaltake Toughpower Grand 750W PSU has detachable cables … so I could have one cable less in my case :] <sarcasm/>

I generally like to build … everything – that means also building computers … but one thing always pisses me off when I build a PC – the cables from the case buttons to motherboard.

Back in 2006 when I was building a gaming PC for my friend it looked more or less like that below – starting a PC with a screw 🙂

Difference

The difference in CPU power between older AMD Ryzen 7 1700 and AMD Ryzen 7 PRO 4750GE is about 25% percent. When it comes to GPU between 5700XT 8GB and 7700XT 12GB its somewhere between 40-60% percent.

There is almost zero ‘visual’ difference between the two – first the older one.

… and the newer one.

FreeBSD

The most important part of this article is sharing that all of these parts – especially GPU – works very well with FreeBSD with drm-kmod and x11-drivers/xlibre-xf86-video-amdgpu XLibre AMD Radeon GPU driver. After I learned how much some Red Hat people wanted to cripple and kill the X11 … I am not going back to Xorg X11 implementation … the XLibre X11 is my home … and do not even get me started on Wayland – the details about that topic are in the 200 MB RAM FreeBSD Desktop article.

Firmware

After install and reboot I started the fwget(8) command so that all needed firmware for FreeBSD will be fetched.

challenger # fwget -v
Trying to match device 0x747e in class video and vendor amd with pci_video_amd
Trying to match device 0x8168 in class network and vendor realtek with pci_network_realtek
Trying to match device 0x24fb in class network and vendor intel with pci_network_intel
Needed firmware packages: 'wifi-firmware-iwlwifi-kmod-7000'                                                    

But as it seams it was not enough as I got these boot errors on every FreeBSD system start.

iwmbtfw: iwmbt_fw_read: open: /usr/local/share/iwmbt-firmware/ibt-hw-37.8.bseq: No such file or directory
iwmbtfw: main: Firmware download failed!                      

… so I start to dig where the problem is. For a start I checked all /etc/rc.d services for iwmbtfw name.

challenger # grep -r iwmbtfw /etc
/etc/devd/iwmbtfw.conf: action "/usr/sbin/iwmbtfw -d $cdev -f /usr/local/share/iwmbt-firmware";

challenger # ls /usr/local/share/iwmbt-firmware
ls: /usr/local/share/iwmbt-firmware: No such file or directory

So the /usr/local/share/iwmbt-firmware seems to be missing but its suppose to be there …

challenger # man iwmbtfw | grep firmware
     Firmware files are available in the comms/iwmbt-firmware port.

OK … lets install the comms/iwmbt-firmware package.

challenger # pkg search -o iwmbt
comms/iwmbt-firmware           Intel Wireless bluetooth adaptor firmwares used by iwmbtfw(8)

challenger # pkg install iwmbt-firmware
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-ports-kmods repository catalogue...
FreeBSD-ports-kmods repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        iwmbt-firmware: 20251111 [FreeBSD-ports]

Number of packages to be installed: 1

The process will require 15 MiB more space.
5 MiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching iwmbt-firmware-20251111: 100%  4840 KiB 619.5 kB/s    00:08
Checking integrity... done (0 conflicting)
[1/1] Installing iwmbt-firmware-20251111...
[1/1] Extracting iwmbt-firmware-20251111: 100%

After reboot I never saw that error message again – so that is the needed fix.

Hardware

This is the exact hardware I got with the prices I paid for it in EUR currency. Some were bought as new and some as used.

 TYPE  EUR  WHAT                                    
 CASE   55  Silverstone SG13
  PSU   55  Thermaltake Toughpower Grand 750W
 MOBO  100  ASRock B550M-ITX/ac
  CPU  145  AMD Ryzen 7 4750GE 8C/16T 35W TDP
  GPU  340  ASRock ATI Radeon 7700XT 12GB Challenger
PASTE   10  Grizzly Thermal Kryonaut 2g
 COOL   40  Thermalight AXP-90 X47
  FAN    5  Savio BLADEX1 120mm PWM
  SSD   45  ADATA SU650 512GB
 NVME   50  Phison ESO512GYLCT-EP3-2L 512GB
  RAM  150  2 x 16GB Goodram DDR4 3600MHz 1.35V CL18
PRICE  995  TOTAL

Not sure it was worth it but I did it anyway.

In the mean time I also upgraded my older HP E221c FullHD monitor for HP E271i FullHD one … with additional HP soundbar mounted below the screen. To be honest – nothing fancy about that – the monitor was about 70 EUR and about 15 EUR for the soundbar.

XLibre X11

I installed these packages.

challenger # \
  pkg install -y \
    drm-kmod \
    xlibre \
    openbox \
    xterm \
    xinit \
    mesa-demos \
    scrot \
    radeontop \
    screen \
    lsblk \
    beadm \
    smartmontools

Next I started the XLibre X11 as usual.

vermaden@challenger % echo openbox > ~/.xinitrc

vermaden@challenger % xinit -- -dpi 75 -nolisten tcp

… and the X11 XLibre display server started as usual – but after I checked the logs … I was using the software scfb driver … and I was not in the video group – so I added myself there.

challenger # pw groupmod video -m vermaden

Next … I searched for amdgpu driver …

challenger # pkg search -o amdgpu
x11-drivers/xf86-video-amdgpu         X.Org amdgpu display driver
x11-drivers/xlibre-xf86-video-amdgpu  XLibre amdgpu display driver

Its important to install the x11-drivers/xlibre-xf86-video-amdgpu one because the other one will force You to uninstall XLibre and install Xorg.

Next I added the AMD Radeon X11 config.

challenger # cat /usr/local/etc/X11/xorg.conf.d/card.conf 
Section "Device"
  Identifier "Card0"
  Driver "amdgpu"
  Option "TearFree" "true"
EndSection

Nothing fancy but works as desired.

I also checked the /var/log/Xorg.0.log for details and …

[2026-09-05 14:48:22] (II) LoadModule: "amdgpu"
[2026-09-05 14:48:22] (II) Loading /usr/local/lib/xorg/modules/xlibre-25/drivers/amdgpu_drv.so
[2026-09-05 14:48:22] (II) Module amdgpu: vendor="X.Org Foundation"
[2026-09-05 14:48:22]   compiled for 1.25.1.9, module version = 25.1.2
[2026-09-05 14:48:22]   Module class: X.Org Video Driver
[2026-09-05 14:48:22]   ABI class: X.Org Video Driver, version 28.1
[2026-09-05 14:48:22] (II) AMDGPU: Driver for AMD Radeon:
        All GPUs supported by the amdgpu kernel driver

Yep … this is what we wanted.

This is how that GPU looks like in the pciconf(8) output.

challenger # pciconf -lv vgapci0
vgapci0@pci0:3:0:0:     class=0x030000 rev=0xff hdr=0x00 vendor=0x1002 device=0x747e subvendor=0x1849 subdevice=0x5323
    vendor     = 'Advanced Micro Devices, Inc. [AMD/ATI]'
    device     = 'Navi 32 [Radeon RX 7700 XT / 7800 XT]'
    class      = display
    subclass   = VGA

This is how resources usage looks like in X11 XLibre session on FreeBSD.


challenger # top -b -ores
last pid:  5702;  load averages:    0.08,    0.08,    0.02  up 0+00:30:16    14:35:51
34 processes:  1 running, 33 sleeping
CPU:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.9% idle
Mem: 92M Active, 101M Inact, 1512M Wired, 2056K Buf, 29G Free
ARC: 141M Total, 24M MFU, 113M MRU, 388K Anon, 670K Header, 2159K Other
     88M Compressed, 207M Uncompressed, 2.35:1 Ratio
Swap: 4096M Total, 4096M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
90382 vermaden     16   0    0   319M   136M select   6   0:01   0.00% Xorg
92537 vermaden      2   0    0    50M    22M select  12   0:00   0.00% openbox
 1117 vermaden      1   0    0    26M    14M select  13   0:00   0.00% xterm
77479 root          1   0    0    25M    10M select  14   0:00   0.00% sshd
17882 root          1   0    0    15M  4356K select   9   0:00   0.00% devd
89878 vermaden      1   1    0    16M  3808K wait     5   0:00   0.00% xinit
 2151 vermaden      1   0    0    14M  3500K wait     2   0:00   0.00% sh
 5702 root          1   0    0    15M  3492K CPU9     9   0:00   0.00% top
 1924 root          1   0    0    14M  3280K wait     4   0:00   0.00% login

This is how the main FreeBSD /etc/rc.conf config looks like.

challenger # cat /etc/rc.conf
  hostname="challenger.local"
  ifconfig_re0="inet 10.0.0.5/24 up"
  defaultrouter="10.0.0.1"

  kld_list="${kld_list} amdgpu amdtemp"
  syslogd_flags="-ss"
  sshd_enable="YES"
  powerd_enable="YES"
  dumpdev="AUTO"
  zfs_enable="YES"

Far from fancy – just simple stuff.

Power

Power measurement seems similar to older model – but slightly smaller.

POWER  STATE                                     
  47W  IDLE without amdgpu.ko loaded
  39W  IDLE with amdgpu.ko loaded
  45W  BUSY with 16 x 100% CPU Thread(s) @ 1.4GHz
  80W  BUSY with 16 x 100% CPU Thread(s) @ 3.1GHz

The Turbo frequency for the AMD Ryzen 7 4750GE CPU is at 4.3GHz … but I am not sure its used … because for example a CPU in my laptop has two ‘final’ speeds shown as 1801/15000 1800/15000 and the one with additional 1 it the one with Turbo mode … but nothing like that for that AMD Ryzen 7 4750GE CPU.

Also … there is only C1 state supported … which is sad 😦

challenger # sysctl dev.cpu.0
dev.cpu.0.temperature: 47.0C
dev.cpu.0.cx_method: C1/hlt
dev.cpu.0.cx_usage_counters: 259234
dev.cpu.0.cx_usage: 100.00% last 19092us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/0
dev.cpu.0.freq_levels: 3100/3778 1700/1615 1400/1277
dev.cpu.0.freq: 1400
dev.cpu.0.%iommu: 
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=ACPI0007 _UID=0 _CID=none
dev.cpu.0.%location: handle=\_SB_.PLTF.C000
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU

This is how the sensors(8) output looks like on that box.

challenger # sensors

            BATTERY/AC/TIME/FAN/SPEED 
 ------------------------------------ 
               dev.cpu.0.cx_supported: C1/1/0 
                   dev.cpu.0.cx_usage: 100.00% last 10882us
                       dev.cpu.0.freq: 1400 
                hw.acpi.cpu.cx_lowest: C1 
                            powerd(8): running

                  SYSTEM/TEMPERATURES 
 ------------------------------------ 
                dev.cpu.0.temperature: 47.1C
                dev.cpu.1.temperature: 47.1C
                dev.cpu.2.temperature: 47.1C
                dev.cpu.3.temperature: 47.1C
                dev.cpu.4.temperature: 47.1C
                dev.cpu.5.temperature: 47.1C
                dev.cpu.6.temperature: 47.1C
                dev.cpu.7.temperature: 47.1C
                dev.cpu.8.temperature: 47.1C
                dev.cpu.9.temperature: 47.1C
               dev.cpu.10.temperature: 47.1C
               dev.cpu.11.temperature: 47.1C
               dev.cpu.12.temperature: 47.1C
               dev.cpu.13.temperature: 47.1C
               dev.cpu.14.temperature: 47.1C
               dev.cpu.15.temperature: 47.1C

                   DISKS/TEMPERATURES 
 ------------------------------------ 
       smart.ada0.temperature_celsius: 34.0C
              smart.nvme0.temperature: 39.0C

Not great. Not terrible.

PKGBASE Upgrade

As I installed stock 15.1-RELEASE I will now upgrade to latest 15.1-pX release available.

challenger # pkg upgrade -r FreeBSD-base
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
FreeBSD-base is up to date.
Checking for upgrades (53 candidates): 100%
Processing candidates (53 candidates): 100%
The following 52 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        FreeBSD-audit: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-autofs: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-bhyve: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-bsnmp: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-clang: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-clang-dev: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-clibs: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-clibs-dev: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-clibs-dev-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-clibs-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-console-tools: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-ctl: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-jail: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-kernel-generic: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-kernel-generic-dbg: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-lib9p: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-lib9p-dev: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-lib9p-dev-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-lib9p-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-libevent1: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-libevent1-dev: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-libevent1-dev-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-libevent1-lib32: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-mtree: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-natd: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-natd-dev: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-natd-dev-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-natd-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-ntp: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-openssl-dev: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-openssl-dev-lib32: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-openssl-lib: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-openssl-lib32: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-pmc: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-ppp: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-rescue: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-runtime: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-runtime-dev: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-runtime-dev-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-runtime-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-src: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-src-sys: 15.1 -> 15.1p3 [FreeBSD-base]
        FreeBSD-syslogd: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-utilities: 15.1 -> 15.1p2 [FreeBSD-base]
        FreeBSD-utilities-dev: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-utilities-dev-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-utilities-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-zfs-dev: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-zfs-dev-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-zfs-lib: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-zfs-lib32: 15.1 -> 15.1p1 [FreeBSD-base]
        FreeBSD-zoneinfo: 15.1 -> 15.1p2 [FreeBSD-base]

Number of packages to be upgraded: 52

579 MiB to be downloaded.

Proceed with this action? [y/N]: y

… and it all went well. After the upgrade finished I rebooted and everything worked like a charm.

Offtopic

Fast forward half a year later after the build happened in that dining room – my 10 years old son comes to me a month ago and says ‘Hey dad I just wrote this PC Builder game with ChatGPT do You want to play it?’ … and after I checked it I was really shocked … positively. I mean … when I was 10 years old I did not even had my AMIGA 600 … I was playing various games on Pegasus – which is a cheap Polish NES clone … I would get AMIGA 600 when I was more or less like 12 years old (speaking from foggy memories) … and I was still playing games like Sensible World od Soccer or Cannon Fodder instead of creating anything. This is how this (after many iterations with AI) PC Builder game looks like.

Yes – its in Polish – but I believe its just one prompt away from making it English/Polish when needed 🙂

My son also surprised me because one of the times I told him that ‘its not only ChatGPT out there – there are others like Perplexity or Claude or Gemini or … and several times he said ‘Dad I run out of tokens and just copied that I had and put that into other AI and it picked up my work and added the changes I wanted …’ – when I look at my son doing this stuff – being literally a Vibe Coder (still at the beginning of his route but still) … I feel like the schooling system is just one big fucking joke … I mean Mathematics/Physics/English/… are needed and useful … bot others? Optional to say the least … and I felt (and act accordingly) when I was in the school. Its just such a fucking waste of time to cram for some exam and forgot all that shit to get a grade.

What I find funny is that my son never played a game I wrote … and I already played two games of his … I wish every parent to have the same.

EOF
Top

Valuable News – 2026/08/31

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

Bhyve Inside Jail: FreeBSD Feature Nobody Uses.
https://blog.hofstede.it/bhyve-inside-a-jail-the-freebsd-feature-nobody-uses/

FreeBSD 14.5-RC1 Now Available.
https://lists.freebsd.org/archives/freebsd-stable/2026-August/004288.html

FreeBSD 14.5-RC1 Released with Several Security Fixes.
https://phoronix.com/news/FreeBSD-14.5-RC1

Why is This Running witr(1) Tool to Trace any Process/Port/Container/File.
https://github.com/pranshuparmar/witr

GSoC 2026 Reports: Improving RAIDframe on NetBSD.
https://blog.netbsd.org/tnf/entry/gsoc2026_raidframe

Install Void Linux with ZFS Boot Environments/Hibernation/Encryption.
https://it-notes.dragas.net/2025/12/22/void-linux-zfs-hibernation-guide/

VirtIO GPU QEMU/UTM Display Driver for FreeBSD Guests so Xorg/Xlibre Use xf86-video-scfb Dirver.
https://gitlab.com/tgasiba/virtio_gpu_qemu

Powercool 1000VA and 850VA UPS on FreeBSD with NUT.
https://forums.sheridancomputers.com/t/powercool-1000va-and-850va-ups-on-freebsd-with-nut/185

New FreeBSD Compatible wifi-tui Tool to Scan/Conenct/Manage WiFi Networks.
https://github.com/hirechbaghdad/wifi-tui/

GParted or Partition Manager for FreeBSD.
https://forums.freebsd.org/threads/gparted-partition-manager.103693/

There is No Linux Admin.
https://vivianvoss.net/blog/no-linux-admin

FreeBSD facl(8) Tool to Manage NFSv4 ACLs.
https://github.com/olgeni/facl

FreeBSD zfs-allow(8) Tool to Manage ZFS Delegated Administration.
https://github.com/olgeni/zfs-allow

FreeBSD zfs-set(8) Tool to Manage ZFS Dataset Properties.
https://github.com/olgeni/zfs-set

FreeBSD zpool-set(8) Tool to Manage ZFS Pool Properties.
https://github.com/olgeni/zpool-set

Hardware Acceleration on FreeBSD 14.4 i386.
https://reddit.com/r/freebsd/comments/1vyez95/hardware_acceleration_on_freebsd_144_i386/

Run OpenBSD on DigitalOcean for $4 per Month.
https://nil.wallyjones.com/run-openbsd-on-digitalocean-for-4month/

What Happens When You Connect to Starbucks WiFi.
https://nil.wallyjones.com/what-happens-when-you-connect-to-starbucks-wi-fi/

Sylve 0.3.0 Released.
https://github.com/AlchemillaHQ/Sylve/releases/tag/v0.3.0

What Shell am I Using?
https://nil.wallyjones.com/what-shell-am-i-using/

FreeBSD Foundation Intern Jim Huang Chen on Raspberry Pi Support and Open Source.
https://freebsdfoundation.org/blog/freebsd-foundation-intern-jim-huang-chen-on-raspberry-pi-support-and-open-source/

Sylve – Modern Virtualization Solution.
https://tomsitcafe.com/2026/07/24/sylve-a-modern-virtualization-solution/

Postgres pgbot(1) Intelligence for AI Agents and Apps.
https://github.com/pgrundev/pgbot

Lenovo XClarity Administrator in FreeBSD Bhyve VM.
https://vermaden.wordpress.com/2026/08/28/lenovo-xclarity-administrator-in-freebsd-bhyve-vm/

BSDnas is Community Fork that Continues TrueNAS CORE on Current FreeBSD.
https://bsdnas.com/

WebZFS is Modern Web Interface for ZFS Administration.
https://webzfs.com/

FreeCORE Descends from FreeNAS/TrueNAS CORE and Builds on FreeBSD and OpenZFS.
https://freecore.org/

Challenges in Bringing AMD ROCm to FreeBSD.
https://phoronix.com/news/AMD-ROCm-FreeBSD-Intern-Report

f3s: Kubernetes with FreeBSD – Part 10 – New Home.
https://foo.zone/gemfeed/2026-09-01-f3s-kubernetes-with-freebsd-part-10.html

Sandbox Inside the Jail.
https://vivianvoss.net/blog/sandbox-inside-the-jail

What Isolation Actually Costs.
https://vivianvoss.net/blog/isolation-costs

BSD Weekly – Issue 293.
https://bsdweekly.com/issues/293

Enabling TRIM on LUKS Encrypted SSD on Linux.
https://forums.sheridancomputers.com/t/enabling-trim-on-a-luks-encrypted-ssd-on-arch-linux/186

UNIX/Audio/Video

BSD Now 678: Ceiling Cat Has Another Meaning.
https://www.bsdnow.tv/678

Hardware

PINE64 Partially Halts Production Due to Memory Shortage.
https://heise.de/en/news/PINE64-partially-halts-production-due-to-memory-shortage-11423218.html

Commodore 77.
https://commodore.net/store/commodore64/

MNT Station Modular and Open Hardware Desktop/Server Computer.
https://crowdsupply.com/mnt-research/mnt-station

Motorola GrapheneOS Phones Will Launch in 2027 Pricier than Pixels
https://arstechnica.com/gadgets/2026/08/motorolas-grapheneos-phones-will-launch-in-2027-priced-higher-than-pixels/

Samsung New P9 USB4 SSD Promises 4000MB/s.
https://dpreview.com/news/step-aside-t9-samsungs-new-p9-usb4-drive-delivers-4-000mb-s-for-photo-workflows/

Life

How Much of Internet is Written With AI?
https://pewresearch.org/data-labs/2026/08/20/how-much-of-the-internet-is-written-with-ai/

Meet Eliza Zhang – Book Scanner at Internet Archive.
https://blog.archive.org/2021/02/09/meet-eliza-zhang-book-scanner-and-viral-video-star/

Temperature Zero for Culture: Why Everything is Starting to Look the Same.
https://laurenleek.substack.com/p/temperature-zero-for-culture-why

Other

Heroes ]|[ Remake – Gameplay Overview Trailer with Commentary.
https://youtube.com/watch?v=i9RahV1tj50

ReactOS 0.4.16 Released with New Graphical Installer and Better Hardware Support.
https://phoronix.com/news/ReactOS-0.4.16-Released

Aptivi Weekly Tech News 2026/08/29.
https://officialaptivi.wordpress.com/2026/08/29/aptivi-weekly-tech-news-issue-13-week-35-8-29-2026/

Lazy Reading for 2026/08/30.
https://dragonflydigest.com/2026/08/30/lazy-reading-for-2026-08-30/

Making My Dream City Builder – Part 1 – Parcels.
https://youtube.com/watch?v=4heM4MqK82w

Making My Dream City Builder – Part 2 – Traffic.
https://www.youtube.com/watch?v=BoqK5wOGpfU

Making My Dream City Builder – Part 3 – City Simulation.
https://www.youtube.com/watch?v=2NvNp9QXMCM

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Expat 2.8.4 released, fixes 4 vulnerabilities

Post by Sebastian Pipping via Hartwork Blog »

For readers new to Expat:

libexpat is a fast streaming XML parser. Alongside libxml2, Expat is one of the most widely used software libre XML parsers written in C, specifically C99. It is cross-platform and licensed under the MIT license.

Expat 2.8.4 was released a few hours ago. The key motivation for cutting a release and doing so now was getting the fixes to four vulnerabilities…

…into the hands of the community.

The vulnerabilities were reported by Darren Carreras, Fabian Wahle (Hap Security), Sorrashut Kaewtaworn, Zeyou Liu, and the fixing was done by Darren Carreras, Sorrashut Kaewtaworn, Zeyou Liu, and me — thank you!

Issue CVE-2026-66046 is worth illustrating, and the fixing pull request contains an attack payload generator in its description. The issue was that if an attacker dials up a document like…

<!DOCTYPE e [
  <!ATTLIST e a0 NMTOKEN "x">
  <!ATTLIST e a1 NMTOKEN "x">
  <!ATTLIST e a2 NMTOKEN "x">
]>
<e a0=" v " a1=" v " a2=" v " />

…from 3 attributes to thousands, they could leverage the now-past quadratic runtime nature of Expat's related machinery to cause denial of service even with moderatly sized payload. The fix was migrating from an O(n) loop to an amortized O(1) hash table lookup.

As usual, there is also non-security work that went into this release. For instance, on the infrastructure side of things, the CI now covers Fil-C, RISC-V, Clang-based MinGW, and (the big-endian architecture) s390x for the first time.

It should be noted that it was the Open Source Sabbatical of the City of Munich that allowed me to focus on libexpat for the community through its funding — thank you! This has been the first month, and there are hopefully five more to come; there continues to be plenty to do. Wish me luck!

Thanks to everyone who contributed to this release of Expat!

For more details about this release, please check out the change log.

If you maintain Expat packaging, a bundled copy of Expat, or a pinned version of Expat, please update to version 2.8.4. Thank you!

Sebastian Pipping

Top

Lenovo XClarity Administrator in FreeBSD Bhyve VM

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

This is another entry that is sponsored by fme AG company. If You have multiple Lenovo ThinkServer systems to manage You may do that by hand with entering UEFI/BIOS setup and doing needed things by hand … or You can setup a Lenovo XClarity Administrator VM.

The problem is that this VM from Lenovo comes by default in three versions … and none of them is by default suited for FreeBSD Bhyve hypervisor.

  • VHD for Windows Hyper-V hypervisor.
  • OVA for VirtualBox and others that support OVA format.
  • QCOW2 for Linux KVM based hypervisors.

As usual I decided not to give up at that point and tried harder … and that trying led me to some interesting conclusions.

I downloaded the QCOW2 version listed as lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2 file – its about 3.2 GB in size.

Then I created the Bhyve VM for that Linux system.

freebsd # vm create lxca 

As I had the Lenovo image in QCOW2 format I used qemu-img(1) from the emulators/qemu package to convert it to RAW format that is suitable for FreeBSD Bhyve hypervisor. One important thing to note before doing the conversion … the size of RAW disk will be 192 GB. It will not matter on ZFS with compression enabled – but it can fill Your disk if You have compression disabled.

freebsd # zfs get -d 0 compression
NAME   PROPERTY     VALUE           SOURCE
zroot  compression  lz4             local

freebsd # du -smh \
            lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2 \
            /vm/lxca/disk0.img 
3.2G    lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2
4.6G    /vm/lxca/disk0.img

freebsd # du -smhA \
              lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2 \
              /vm/lxca/disk0.img
3.2G    lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2
192G    /vm/lxca/disk0.img

Now the conversion.

freebsd # pkg which -o $( which qemu-img )
/usr/local/bin/qemu-img was installed by package emulators/qemu

freebsd # \
  qemu-img convert -f qcow2 -O raw \
    lnvgy_sw_lxca_120-4.3.0_kvm_x86-64.qcow2 \
    /vm/lxca/disk0.img

I then investigated what actually is on that raw disk and FreeBSD md(4) driver helped here beautifully. We will also use my mdconfig.sh script to make it easier.

freebsd # mdconfig.sh -c disk0.img
IN: created vnode at /dev/md0

freebsd # lsblk md0
DEVICE         MAJ:MIN  SIZE TYPE                                    LABEL MOUNT
md0              1:183  192G MBR                                         - -
  <FREE>         -:-    993K -                                           - -
  md0s1          1:184   24G linux-data                         ext2fs/ELR -
  md0s2          1:205    8G linux-data                 ext2fs/ELR_VAR_LOG -
  md0s3          1:206  160G linux-data               ext2fs/ELR_LXCA_DATA -

So … now we know that its an MBR partitioning scheme … which means UEFI boot is not possible – and this is where it gets ugly because Bhyve is not VirtualBox (and I admit this is one of the Bhyve downsides) which means we now have to find needed parameters for Bhyve to boot that Linux VM.

We can at least mount these partitions to check what is in them … and also check the GRUB config. We will use lklfuse(8) from the filesystems/lkl package for that – lets try with the 1st partition.

freebsd # pkg which -o $( which lklfuse ) 
/usr/local/bin/lklfuse was installed by package filesystems/lkl

freebsd # mkdir -p /mnt/tmp

freebsd # lklfuse -o type=ext4 /dev/md0s1 /mnt/tmp

freebsd # ls -l /mnt/tmp
total 76
drwxr-xr-x   2 root wheel     4096 Feb 19  2025 bin
drwxr-xr-x   3 root wheel     4096 Apr  9  2025 boot
lrwxrwxrwx   1 root wheel       28 Jun  6  2025 chroot -> /opt/lenovo/lxca/data/chroot
drwxr-xr-x   2 root wheel     4096 Feb 19  2025 dev
lrwxrwxrwx   1 800  800         26 Jun  6  2025 dump -> /opt/lenovo/lxca/data/dump
drwxr-xr-x  66 root wheel     4096 Aug 28 00:40 etc
drwxr-xr-x  12 root wheel     4096 Aug 28 00:40 home
drwxr-xr-x   8 root wheel     4096 Feb 19  2025 lib
lrwxrwxrwx   1 root wheel        3 Jun  6  2025 lib64 -> lib
drwx------   2 root wheel    16384 Jun  6  2025 lost+found
drwxr-xr-x   2 root wheel     4096 Feb 19  2025 media
drwxrwxr-x   2 root wheel     4096 Feb 19  2025 mnt
drwxr-xr-x   7 root wheel     4096 Jun  6  2025 opt
dr-xr-xr-x   2 root wheel     4096 Feb 19  2025 proc
drwxr-xr-x  10 root wheel     4096 Jun  6  2025 run
drwxr-xr-x   2 root wheel     4096 Feb 19  2025 sbin
dr-xr-xr-x   2 root wheel     4096 Feb 19  2025 sys
lrwxrwxrwx   1 root wheel        8 Jun  6  2025 tmp -> /var/tmp
drwxr-xr-x  11 root wheel     4096 Feb 19  2025 usr
drwxr-xr-x  11 root wheel     4096 Jun  6  2025 var

freebsd # ls -l /mnt/tmp/boot
total 14176
lrwxrwxrwx   1 root wheel       16 Jun  6  2025 bzImage -> bzImage-5.15.178
-rw-r--r--   1 root wheel 11837760 Feb 19  2025 bzImage-5.15.178
drwxr-xr-x   2 root wheel     4096 Jun  6  2025 grub
-rw-r--r--   1 root wheel  2670312 Feb 19  2025 initrd
freebsd # cat /mnt/tmp/boot/grub/grub.conf
default=0
timeout=1

title System Management Platform
        root (hd0,0)
        kernel /boot/bzImage ro console=tty0 console=ttyS0 root=LABEL=ELR clocksource_failover=acpi_pm net.ifnames=0 clocksource_failover=acpi_pm
        initrd /boot/initrd

freebsd # cd /

freebsd # umount /mnt/tmp

freebsd # mdconfig.sh -d 0
IN: deleted vnode at /dev/md0

So … we know what the GRUB config looks like – lets try to translate that into FreeBSD Bhyve VM config. After playing some time I settled on this lxca VM config as shown below.

freebsd # env EDITOR=cat vm config lxca
loader="grub"
grub_run_partition="msdos1"
grub_run_dir="/boot/grub"
grub_run0="linux /boot/bzImage ro console=tty0 console=ttyS0 root=LABEL=ELR clocksource_failover=acpi_pm net.ifnames=0 clocksource_failover=acpi_pm"
grub_run1="initrd /boot/initrd"
cpu=3
memory=16G
network0_type="e1000"
network0_switch="public"
network0_mac="58:9c:fc:06:2e:71"
network1_type="e1000"
network1_switch="public"
network1_mac="58:9c:fc:02:3b:c0"
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="53211072-22f1-43f0-97c4-fae86903b2fc"

Then we need to start the VM with usual command.

freebsd # vm start lxca

You may (and should) trace the boot process with vm console {VM} command of course.

freebsd # vm console lxca

You will see lots of Linux boot messages both first from the Linux kernel itself and later from the booting process of applications/daemons/databases needed to start Lenovo XClarity Administrator appliance. After some time – definitely longer then shorter – You will see something like that below.

**************************************************************
 This interface is not for user or customer usage    *********
**************************************************************


------------------------------------------
Lenovo LXCA - Version 4.3.0 build 120
------------------------------------------

eth0: flags=4163  mtu 1500
        inet 10.1.1.52  netmask 255.255.255.0  broadcast 10.1.1.255
        inet6 fe80::5a9c:fcff:fe06:2e71  prefixlen 64  scopeid 0x20
        ether 58:9c:fc:06:2e:71  txqueuelen 1000  (Ethernet)
        RX errors 0  dropped 0  overruns 0  frame 0

eth1:      Disabled

localhost login: 

This means that You can how try to access https://10.1.1.52/ in Your browser … and after I tried that I (of course) needed to accept the self generated certificate and then I saw this.

Voila! It works! 🙂

Now You can configure it from the browser as needed.

Guide that really helped me to setup all these things right was the Lenovo XClarity Administrator – Users Guide that has this in its contents.

It shows TWO network interfaces here – eth0 and eth1 – not just one – and when I used only one – everything failed – after I added 2nd network interface – everything started to work as desired.

I do not have anything more to add here – I am really glad that I persisted hard to try all available options here – take care 🙂

UPDATE 1 – Lenovo XClarity One

Graham Perrin commented on one of the places where I shared the article that XClarity Administrator will not be supported after 2027/06/30 and that XClarity One is now the future … so I tried to setup XClarity One under FreeBSD Bhyve VM and this update will show You the details.

 

The Lenovo XClarity One can be downloaded as QCOW2 image here – so I did. Now it seams that there are TWO disks instead of just ONE – but that does not bring any problems for Bhyve – we will attach them both.

freebsd # vm create lxco

freebsd # cd /vm/lxco

freebsd # du -smh ~/lnvgy_sw_xc1_26p.2.0_kvm_indiv.tgz                       
5.0G    /home/vermaden/lnvgy_sw_xc1_26p.2.0_kvm_indiv.tgz

freebsd # du -smhA ~/lnvgy_sw_xc1_26p.2.0_kvm_indiv.tgz
5.0G    /home/vermaden/lnvgy_sw_xc1_26p.2.0_kvm_indiv.tgz

freebsd # tar -xvf ~/lnvgy_sw_xc1_26p.2.0_kvm_indiv.tgz 
x ./lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk1.qcow2
x ./lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk2.qcow2

freebsd # qemu-img convert -f qcow2 -O raw lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk1.qcow2 disk0.img

freebsd # qemu-img convert -f qcow2 -O raw lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk2.qcow2 disk1.img

freebsd # ls -l /vm/lxco
total 13534815
-rw-r--r--  1 root wheel 257699151872 Aug 28 21:27 disk0.img
-rw-r--r--  1 root wheel 536870912000 Aug 28 21:25 disk1.img
-rw-r--r--  1 root wheel  16380461056 Jul 16 13:58 lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk1.qcow2
-rw-r--r--  1 root wheel       204608 Jul 16 13:58 lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk2.qcow2
-rw-r--r--  1 root wheel          213 Aug 28 21:18 lxco.conf
-rw-r--r--  1 root wheel           66 Aug 28 21:18 vm-bhyve.log

freebsd # ls -lh /vm/lxco
total 13534815
-rw-r--r--  1 root wheel 240G Aug 28 21:27 disk0.img
-rw-r--r--  1 root wheel 500G Aug 28 21:25 disk1.img
-rw-r--r--  1 root wheel  15G Jul 16 13:58 lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk1.qcow2
-rw-r--r--  1 root wheel 200K Jul 16 13:58 lnvgy_sw_xc1_7909-uat_kvm_x86_64_disk2.qcow2
-rw-r--r--  1 root wheel 213B Aug 28 21:18 lxco.conf
-rw-r--r--  1 root wheel  66B Aug 28 21:18 vm-bhyve.log

I believe at least some things could have change – so lets have a look inside 🙂

freebsd # mdconfig.sh -c disk0.img
IN: created vnode at /dev/md0

freebsd # lsblk md0
DEVICE         MAJ:MIN  SIZE TYPE                                    LABEL MOUNT
md0              1:182  240G MBR                                         - -
           -:-    993K -                                           - -
  md0s1          1:183    2G linux-data                     gpt/linux-data -
  md0s2          1:184   20G linux-data                     gpt/linux-data -
  md0s3          1:206   20G linux-data                     gpt/linux-data -
  md0s4          2:7    198G EBR                                   gpt/ebr -
    md0s4+00000001   2:9     99G linux-data                                  - -
    md0s4+03295525   2:11    99G linux-data                                  - -
           -:-     62K -                                           - -

freebsd # lklfuse -o type=ext4 /dev/md0s1 /mnt/tmp

freebsd # ls -l /mnt/tmp
total 16400
-rw-r--r--  1 root wheel 16749568 Oct  9  2025 bzImage
drwxr-xr-x  3 root wheel     4096 Oct  9  2025 grub
drwx------  2 root wheel    16384 Oct  9  2025 lost+found

freebsd # ls -l /mnt/tmp/grub
total 16
-rw-r--r--  1 root wheel   129 Oct  9  2025 grub.cfg
drwxr-xr-x  2 root wheel 12288 Oct  9  2025 i386-pc

freebsd # cat /mnt/tmp/grub/grub.cfg
set timeout=10
  menuentry 'XClarity Portal' {
  linux /bzImage root=PARTUUID=f869306a-02 rw net.ifnames=0 loglevel=4 console=tty1
}

So … no more initrd and different linux line … not much. This is how the new vm-bhyve config looks like.

freebsd # cat /vm/lxco/lxco.conf
loader="grub"
grub_run_dir="/grub"
grub_run_partition="msdos1"
grub_run0="linux /bzImage root=PARTUUID=f869306a-02 rw net.ifnames=0 loglevel=4 console=tty1"
cpu=8
memory=16G
network0_type="e1000"
network0_switch="public"
network0_mac="58:9c:fc:0a:cf:eb"
network1_type="e1000"
network1_switch="public"
network1_mac="58:9c:fc:03:25:d7"
disk0_type="virtio-blk"
disk0_name="disk0.img"
disk1_type="virtio-blk"
disk1_name="disk1.img"
uuid="d2eb04ed-1884-45c8-987f-f5df7e412664"

Lets start this thing.

freebsd # vm start lxco

freebsd # vm console lxco

Seems to boot without errors – this is what is visible on the console.

(...)

        Welcome to XClarity One Portal 26p.2.0-7909

VM Information:
---------------
-                IPv4: 
-             Netmask: 0.0.0.0
-             Gateway: 
-        Internal CNI: 192.168.255.0/25
-                UUID: 89705D12214F40249DB75E00D2D1C522
System Information:
-------------------
-         CPU # Cores: 8
-     CPU Utilization: 28.83 %
-  Memory Utilization: 2.31 % (370 MB of 15999 MB)
- Storage Utilization: 47.25 % (113.41 GB of 240 GB)

xc1p login: 

        Welcome to XClarity One Portal 26p.2.0-7909

VM Information:
---------------
-                IPv4: 10.1.1.36
-             Netmask: 255.255.255.0
-             Gateway: 10.1.1.1
-        Internal CNI: 192.168.255.0/25
-                UUID: 89705D12214F40249DB75E00D2D1C522
System Information:
-------------------
-         CPU # Cores: 8
-     CPU Utilization: 18.96 %
-  Memory Utilization: 6.77 % (1083 MB of 15999 MB)
- Storage Utilization: 47.25 % (113.41 GB of 240 GB)

xc1p login: 

It generally ‘refreshes’ or just ‘adds’ new CPU/RAM/… status once in a while. This is how it looks like in the web browser (after accepting self signed certificate warning). As You see it takes a while for it to pick up the DHCP IP address.

 

Seems to work … till next time.

EOF
Top

MMO-2044 V2: Allianz-System – Gemeinsam gegen den Schwarm!

Post by Bernd Dau via Zockertown: Nerten News »

 

🚀 MMO-2044 V2 — Allianz-System ist da! 🚀

Wie wäre es, wenn du nicht allein gegen eine übermächtige Bedrohung kämpfst?

In MMO-2044 V2 schmiedest du jetzt Allianzen mit anderen Spielern, teilst Wissen und koordinierst Angriffe, denn nur gemeinsam können wir den Schwarm besiegen!

🆕 Neues Allianz-System:

  • Gründe deine eigene Allianz mit Name, Beschreibung und Motto
  • Lade Spieler ein oder trete öffentlichen Allianzen bei
  • Verwalte Mitglieder als Leader oder Officer
  • Teile Wissen – Allianz-Mitglieder sehen alle gemeinsamen Forschungen

🎯 In der nächsten Version kommen:

  • 🎯 Koordinierte Flottenangriffe (Synchronisiertes Eintreffen ±2 Minuten)
  • 🔍 Spionage-Sonden für Schwarm-Erkundung
  • 📜 Progressive Enthüllung der Schwarm-Bedrohung
📢 Beta-Test starten! Die erste Beta mit Allianz-System steht kurz vor der Veröffentlichung!
Wann? Nächstes Wochenende
Teste die Allianz-Funktionen und melde Bugs!

Bleibt dran – der Schwarm kommt! 🪐

Top

FreeBSD 14.5-RC1 Available

Post by FreeBSD Newsflash via FreeBSD News Flash »

The first Release Candidate build for the FreeBSD 14.5 release cycle is now available. ISO images for the amd64, i386, armv7, aarch64, powerpc, powerpcspe, powerpc64, powerpc64le, and riscv64 architectures are FreeBSD mirror sites.
Top

FreeBSD backup libraries such as base/FreeBSD-kerberos-lib-backup-libkrb5profile.so

Post by Dan Langille via Dan Langille's Other Diary »

I build my own FreeBSD packages using poudriere. I do this for various reasons, which include:

  • Getting security updates faster because I build nightly
  • Some packages I install use non-default build options

One of the monitoring checks I have in place will identify any installed package which is not in the package repository. That helps me identify any failed package builds or packages I have build once, installed, and did not added to my package building system. You can find that check in my git repo of nagios things.

Case in point:

[12:33 r730-01 dvl ~] % pkg version -vRL= | grep orphaned
FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-20260717191016 ?   orphaned: base/FreeBSD-kerberos-lib-backup-libkrb5profile.so.122
FreeBSD-lldb-backup-libprivatelldb.so.19-20260717191013 ?   orphaned: base/FreeBSD-lldb-backup-libprivatelldb.so.19
FreeBSD-runtime-backup-libpam.so.6-20260717191009 ?   orphaned: base/FreeBSD-runtime-backup-libpam.so.6
FreeBSD-runtime-backup-libprivatezstd.so.5-20260717191009 ?   orphaned: base/FreeBSD-runtime-backup-libprivatezstd.so.5
lmdb-backup-liblmdb.so.0-20260803225943 ?   orphaned: databases/lmdb-backup-liblmdb.so.0
mesa-libs-backup-libgallium-26.1.3.so-20260803225944 ?   orphaned: graphics/mesa-libs-backup-libgallium-26.1.3.so

I don’t build my own FreeBSD-base packages. There is no reason for that. I might, one day. But not now.

Notice how all these orphaned packages contain backup in their name.

I asked on Mastodon some time ago. I got an answer. It pointed to a man page. I forgot. I posted again. I got an answer. Thanks Lěng Shuāng (冷霜).

I was pointed to man pkg.conf which contains:

     BACKUP_LIBRARIES: boolean
		  If set to true and if an upgrade will remove a library  (typi-
		  cally  due  to the library version number having been bumped),
		  then pkg(8) will back up the library to the  path  defined  by
		  BACKUP_LIBRARY_PATH.	 A  package containing the backed up li-
		  brary will be created.  This package is named after the origi-
		  nal  package	containing  the   library,   with   the   suffix
		  `-backup-libraries'  appended.   The	package  version will be
		  bumped whenever an additional library is backed up.	Default:
		  NO.

OK, let’s check those pkg.conf files.

Yes, most of my hosts had that configuration setting. Not all of the hosts had the orphaned packages problem. However, every one which did have orphaned FreeBSD-base packages, also had that setting.

Clean up on aisle 3

Here I go, cleaning up that host:

[12:36 r730-01 dvl ~] % sudo pkg delete FreeBSD-kerberos-lib-backup-libkrb5profile.so.122 FreeBSD-lldb-backup-libprivatelldb.so.19 FreeBSD-runtime-backup-libpam.so.6 FreeBSD-runtime-backup-libprivatezstd.so.5
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 4 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
	FreeBSD-kerberos-lib-backup-libkrb5profile.so.122: 20260717191016
	FreeBSD-lldb-backup-libprivatelldb.so.19: 20260717191013
	FreeBSD-runtime-backup-libpam.so.6: 20260717191009
	FreeBSD-runtime-backup-libprivatezstd.so.5: 20260717191009

Number of packages to be removed: 4

Proceed with deinstalling packages? [y/N]: y
[1/4] Deinstalling FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-20260717191016...
[1/4] Deleting files for FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-2026[1/4] Deleting files for FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-20260717191016:   0%
FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-20260717191016: missing file /root/usr/local/lib/compat/pkg/libkrb5profile.so.122
[1/4] Deleting files for FreeBSD-kerberos-lib-backup-libkrb5profile.so.122-20260717191016: 100%
[2/4] Deinstalling FreeBSD-lldb-backup-libprivatelldb.so.19-20260717191013...
[2/4] Deleting files for FreeBSD-lldb-backup-libprivatelldb.so.19-2026071719101[2/4] Deleting files for FreeBSD-lldb-backup-libprivatelldb.so.19-20260717191013:   0%
FreeBSD-lldb-backup-libprivatelldb.so.19-20260717191013: missing file /root/usr/local/lib/compat/pkg/libprivatelldb.so.19
[2/4] Deleting files for FreeBSD-lldb-backup-libprivatelldb.so.19-20260717191013: 100%
[3/4] Deinstalling FreeBSD-runtime-backup-libpam.so.6-20260717191009...
[3/4] Deleting files for FreeBSD-runtime-backup-libpam.so.6-20260717191009:   0[3/4] Deleting files for FreeBSD-runtime-backup-libpam.so.6-20260717191009:   0[3/4] Deleting files for FreeBSD-runtime-backup-libpam.so.6-20260717191009: 100%
[4/4] Deinstalling FreeBSD-runtime-backup-libprivatezstd.so.5-20260717191009...
[4/4] Deleting files for FreeBSD-runtime-backup-libprivatezstd.so.5-20260717191[4/4] Deleting files for FreeBSD-runtime-backup-libprivatezstd.so.5-20260717191[4/4] Deleting files for FreeBSD-runtime-backup-libprivatezstd.so.5-20260717191009: 100%

These packages are not FreeBSD-base related and are for me to investigate later.

[12:37 r730-01 dvl ~] % pkg version -vRL= | grep orphaned
lmdb-backup-liblmdb.so.0-20260803225943 ?   orphaned: databases/lmdb-backup-liblmdb.so.0
mesa-libs-backup-libgallium-26.1.3.so-20260803225944 ?   orphaned: graphics/mesa-libs-backup-libgallium-26.1.3.so
[12:37 r730-01 dvl ~] % 

I removed the setting

I am sure I added that configuration setting on my own. It was not a recommendation I had seen. I checked my ansible configuration. That setting is not specified there.

I also ran this query on each host (which runs jails):

[12:52 r730-01 dvl ~] % grep BACKUP /jails/*/usr/local/etc/pkg.conf
[12:53 r730-01 dvl ~] % 
Top

Two Alleged ‘TeamPCP’ Hackers Arrested in Australia

Post by Brian Krebs via Krebs on Security »

Authorities in Australia have arrested two men believed to be members of TeamPCP, a prolific cybercrime and data extortion group blamed for perpetrating the longest running spree of software supply chain attacks ever.

In a statement released today, the Australian Federal Police (AFP) said two men from Western Australia, aged 21 and 23, were arrested in connection with a “sophisticated cybercrime syndicate that allegedly created malicious open-source software to rob thousands of global businesses.”

The AFP did not name the defendants, but KrebsOnSecurity learned the 21-year-old suspect’s real identity in June, and has been communicating with him ever since. This story includes interviews with TeamPCP’s self-described spokesperson, and examines clues left behind by the TeamPCP leader that likely led to his undoing.

TeamPCP vaulted onto the cybercrime scene in late 2025, embedding malicious code in hundreds of open source software tools and extorting victims for profit. Members of the group made headlines by compromising corporate cloud environments using a self-propagating worm dubbed Shai-Hulud, which added malicious code to open source programs maintained by developers whose credentials at public code repositories like GitHub or NPM were phished or stolen.

Writing for Wired, journalist Andy Greenberg described TeamPCP’s core tactic as a kind of cyclical exploitation of software developers.

“The hackers gain access to a network where an open source tool commonly used by coders is being developed,” Greenberg wrote in May. “The hackers plant malware in the tool that ends up on other software developers’ machines, including some who are writing other tools intended to be used by coders. The malware allows TeamPCP’s hackers to steal credentials that let them publish malicious versions of those software development tools, too. The cycle repeats, and TeamPCP’s collection of breached networks grows.”

TeamPCP also has practiced something akin to cyclical recruitment. In May, the source code for the third iteration of Shai-Hulud was published online, and TeamPCP soon after launched a contest offering $1,000 in virtual currency to whichever participant could conduct the largest supply chain operation using the worm’s code. According to the contest rules, participants were scored based on the number of weekly and monthly downloads of packages they compromised — directly incentivizing them to target the most popular code libraries.

A screenshot of a message from TeamPCP’s Telegram account, announcing the supply chain hacking contest. Image: dataminr.com.

“TeamPCP has stated the competition is a recruiting opportunity and they intend to purchase all meaningful access harvested from participants’ campaigns,” the security firm Dataminr wrote. “The $1,000 XMR (Monero) prize is a recruitment floor and has been dismissed by the actor as ‘just like participation trophy,’ adding ‘if you find something good you will be paid way more,’ confirming the contest’s true function as talent identification and malicious access acquisition at scale.”

In March, TeamPCP executed a supply chain attack targeting AI infrastructure by compromising the code for LiteLLM, an open source AI gateway that connects users to more than 100 different large language models. A recent analysis by the security firm CloudSEK found TeamPCPs attack on LiteLLM harvested cloud service keys and other secrets from more than 2,500 organizations, including many of the world’s top technology companies.

In May, TeamPCP claimed credit for compromising at least 3,800 code repositories at the Microsoft-owned GitHub, after a GitHub developer installed a code extension that was compromised by TeamPCP’s malware.

MEET THE CYBERCATS

Security experts say TeamPCP is less of a hacker group than an amalgamation of threat actors from multiple cybercriminal gangs who sometimes work together toward similar goals.

“It is not a structured criminal crew with a single operator,” said Austin Larsen, a principal threat analyst with the Google Threat Intelligence Group. “It is a peer community of individually-skilled actors, with one clear center of gravity.”

That center of gravity is George Prepakis, an accomplished security researcher and self-described exploit developer who operates the Twitter/X profile @kernelstub. Earlier this year, @kernelstub tweeted a public invite link to a Matrix chat server he created and dubbed “Cybercats,” and TeamPCP and several other cybercrime entities have been using this server to communicate daily for the past several months.

A screenshot of the Matrix chat server “Cybercats,” whose members used hacker handles associated with multiple distinct cybercrime groups that have occasionally collaborated on a series of supply chain and data ransom attacks over the past nine months.

Kernelstub, like other administrators in the Cybercats chat, has been using his Twitter/X profile name as his handle in these Matrix communications, frequently tweeting references to other members and to conversations taking place in the Cybercats chat. In a number of cases, the corresponding X accounts for members of the Cybercats chat taunted cybercrime victims publicly before the incidents were reported in the news media.

The Cybercats administrator listed at the top of the screenshot above — “Boxturtle” — is a close associate of TeamPCP who has been tweeting about the group’s conquests under the name @xpl0itrsturtle. This handle corresponds to a data breach broker active on Breachforums and Darkforums who has been selling data stolen in a wave of recent breaches at automobile manufacturers, including BMW Group, Audi, Honda, Mercedes-Benz, Volvo and Toyota, as well as data allegedly taken from Snapchat and SportRadar.

The data leak site for the extortion group or handle “xpl0itrs.”

The Cybercats administrator “SeesawSec” in the screenshot above is the alias of whoever is behind the cybercrime group known as Fulcrumsec, which recently claimed credit for data extortion attacks against the pharmaceutical giant Novo Nordisk, the data broker LexisNexis, and Avnet, a Fortune 500 distributor of electronic components.

The data leak site of Fulcrum Security, a.k.a. Fulcrumsec.

The Cybercats administrator “@pcpcasper” also has been using a similar name on X to discuss TeamPCP’s attacks and victims. This person has an extensive message history on Telegram, where their messages and shared videos show @pcpcasper is an active and vocal member of the National Socialist Network, a neo-Nazi political organization based in Australia.

At one point in these chats, @pcpcasper shared videos and images of what they claimed was their cat, and several of those videos place this user in Western Australia. One source close to the investigation told KrebsOnSecurity that @pcpcasper was one of the two arrested, a claim supported by messages that @kernelstub posted online this morning.

The Cybercats member roster pictured above also features an administrator with the username “T,” which is short for the now-banned Twitter/X profile @pcpcats, the account operated by the self-described TeamPCP spokesperson who was arrested today. As we’ll see in a moment, @pcpcats also is from Western Australia.

By the time @kernelstub tweeted a public invite link to the Cybercats Matrix server, T/@pcpcats was posting only infrequently to the group chat, with other members often inquiring as to his whereabouts and well-being. The group’s collective concern related to @pcpcats’s tendency to blame his increasingly extended absences on the use of hallucinogens and other narcotics that kept him awake for days on end, but also caused him to crash in bed for several days after the highs wore off.

WHO IS THE TEAMPCP LEADER?

The Cybercats member @pcpcats has used multiple nicknames on the cybercrime forums, including EllisD25/LSD on Darkforums, BulkDMT on Breachstars, and Express on Breachforums. These accounts are linked because they all advertised the same Tox ID and/or Session ID as instant message contact handles in their cybercrime forum posts. BulkDMT was also known on the forums as DMT Host, which was a virtual private server (VPS) hosting service that was peddled on Darkforums and Breachstars.

DMT Host/EllisD25, posting on the English-language cybercrime community DarkForums in September 2025. Image: ke-la.com.

According to the cyber intelligence firm Intel 471, Express registered on Breachforums using the email address shitstickpp@gmail.com. Intel 471 finds Express posted on Breachforums across a two-month period in 2025 using four different Internet addresses located in South Africa. On July 30, 2025, Express announced on Breachforums they were selling access to 14 gigabytes of data stolen from South Africa’s State Information Technology Agency.

The threat intelligence platform Flashpoint recorded more than a year’s worth of messages from the TeamPCP leader’s alter ego on Telegram — Persy_PCP —  who claimed they split their life living between two countries [full disclosure: Flashpoint is an advertiser on this blog]. “I have these [files] as well, problem is these are in another country,” Persy_PCP explained to another user inquiring about a stolen data set in November 2025.

Later that month, Persy_PCP complained, “My whole country is racist and they want people like me dead.” Flashpoint records show BulkDMT shared in September 2025 that “this country is going to fucking starve when they take the farmers land,” a likely reference to white landowners in South Africa who claim to be targeted by an ongoing genocide campaign.

This tracks with public reporting on TeamPCP. Cyberscoop reported in June that Google had traced TeamPCP’s residential and mobile Internet address connections to South Africa, “indicating the primary operator was located there during at least some of its attacks.”

BulkDMT also shared on the group chat at Breachforums that they were recovering from an addiction to methamphetamine. “My life is kinda fucked rn [right now], but that’s fine and there isn’t really a point in pouring so much emotional energy into that fact, my parents had money but I unfortunately got really addicted to some things so I don’t get to benefit from that. As long as I continue to survive, stay sober, and move closer towards my goals that’s enough drive and meaning.”

The identity threat protection company SpyCloud finds shitstickpp@gmail.com shows up in the registration of an account called ChristmasSnow on the cybercrime community Raidforums in 2022. Nearly all of the Internet addresses used to access that account came from ISPs in Perth, Australia, SpyCloud found.

KrebsOnSecurity looked up all of those Perth IP addresses in passive DNS records maintained by DomainTools.com, and found one of them — 211.27.196.111 — for several years was used as a private file server by a family in Perth with the last name of Thomson. Those records show at least three hosts — ithomson.direct.quickconnect.to (a remote Synology server), kthomson0061.direct.quickconnect.to, and joshuawthomson39.myqnapcloud.com (a QNAP network storage device) — persisted at that address between 2022 and 2025.

Searching on “joshuathomson39” in the breach tracking service Constella Intelligence reveals an account at the freight forwarding company kwe.com created in the name of Joshua Thomson from Perth, Australia. The open source intelligence platform Epieos finds the phone number attached to that kwe.com account was used to register a Facebook profile for Josh Thomson, which says his family includes a brother named Ruben, his father Ian, and his mom Cindy.

That Facebook profile also says Josh and his family are originally from Pietermaritzburg, in KwaZulu-Natal, South Africa, but currently living in Cottesloe, a beach-side suburb of Perth. A search in DomainTools for Ian Thomson and Australia unearthed five domains by the same registrant, including securecomputing.au, thomson.org.au, and thomsonfamily.net.au. Ian Thomson is a dentist in Cottesloe, and a biography says he graduated from The University of the Witwatersrand in Johannesburg, South Africa.

Constella finds a joshua@thomson.org.au registered a number of accounts online, but Josh doesn’t seem to have much of a connection to dodgy cybercrime forums. His brother Ruben, on the other hand, has quite the presence on these communities, dating back to at least 2018. Constella reports ruben@thomson.org.au frequently reused the password “joshuathomson1,” and Constella further finds that password was used by just a handful of accounts, including yolosolo17@gmail.com and surfinup8@gmail.com.

According to Intel 471, surfinup8@gmail.com was used to register the user Yolosolo17 on the crime forum Altenen in 2018, and that user account was registered from the Perth address 110.141.230.15. On Altenen, Yolosolo17 advertised free web proxies, as well as the domain rubenthomson.com, which was at one point used to sell steeply discounted iPhones. DomainTools says rubenthomson.com was hosted at 110.141.230.15 and registered to surfinup8@gmail.com.

A cached copy of the domain rubenthomson.com from 2017 shows a login page underneath a banded stack of money. Image: archive.org.

SpyCloud reports 10.141.230.15 was used by the email address sheepstealing@gmail.com on Raidforums and surfinup8@gmail.com on Nulled, and that the same IP was used by the email addresses ian@thomsonfamily.net.au, jasper@yakuza.cc, and rubenthomson1@gmail.com. SpyCloud also shows that sheepstealing Gmail address is tied to the accounts Sheep420, YoloSolo117 and Yakuza.cc on Raidforums, and to the account “Sheep Stealing” on Hackforums. Intel 471 says sheepstealing@gmail.com was used to register the account DingoFlour on Breachforums in October 2023, as well Sheepx on Altenen.

Epieos reports that ruben@securecomputing.au is tied to an Airbnb account for Ruben, who described himself as a Web developer who went to school at the University of Western Australia and was living outside the country. “Hey, I’m Ruben, my friends call me Ellis. I’m a Perth creative who occasionally books rooms when visiting family and for photography.”

Epieos also finds sheepstealing@gmail.com registered an upwork.com profile under the name Ruben, who said his main skills are setting up secure server hosting solutions and PHP full-stack Web development.

“I’m familiar with Linux, working with relational databases (SQL),” the Upwork profile reads. “I also script in Python mainly for writing social media bots.”

The Upwork profile for Ruben Thomson in Cottesloe, Australia.

Epieos further discovered sheepstealing@gmail.com is connected to a Microsoft account for Ruben Thomson, and to a now-defunct GitHub account called XmasSnow/XmasSnowisBack that scammed people on the forums in 2022 by claiming to sell exclusive exploits for recently-released software patches (recall that shitstickpp@gmail.com was used to register a forum account named ChristmasSnow).

This same sheepstealing email address registered a Twitter/X account in 2026 called “Gone Fishing” that lists its location as South Africa. That Gmail account also left several reviews for businesses listed on Google Maps over the past seven years, but all of those establishments are located on the west coast of Australia.

Business reviews in Western Australia left by the Google account sheepstealing at gmail.com.

The people search service Pipl finds a 21-year-old Ruben Thomson in Western Australia who has a phone number ending in 979. A lookup on that number at Epieos reveals it is connected to a TikTok account under the name Ellis, and to a PayPal account in the name of Ruben Thomson.

Finally, a search on the name Ruben Thomson from Cottesloe at the Australian government’s record of registered businesses finds he has incorporated or served as an official in multiple companies created since 2024, including Secure Computing Solutions, Tensor Industries, and another entity ironically named OPSEC Express. Recall that Express was BulkDMT’s nickname on Breachforums.

Australian companies connected to Ruben Thomson. Image: abr.business.gov.au.

It’s ironic because OPSEC is short for the term “operational security,” which refers to techniques and behaviors used to obfuscate and compartmentalize one’s real-life identity online, and using your cybercrime handle as part of your own company name is very much the antithesis of that practice.

There is at least one other major opsec failure by Ruben that exposed a link to TeamPCP. In June 2025, someone using the name Ruben Thomson registered on HackerOne, a popular “bug bounty” program that seeks to reward and recognize researchers who agree to work with affected software vendors to help fix the flaws before publishing about their findings. What was Ruben Thomson’s chosen HackerOne username? Deadcatx3, a nickname that has been flagged by multiple security firms as an alias used by TeamPCP.

The HackerOne profile for “Ruben Thomson” uses the nickname Deadcatx3, which multiple security firms have concluded is an alias used by TeamPCP. Image credit: flare.io.

INTERVIEW WITH ELLIS

In early July 2026, not long after having discovered clues about Ellis’s real life identity, KrebsOnSecurity interviewed the TeamPCP leader via Signal, where he was remarkably open about his activities and personal struggles [for the sake of simplicity, the TeamPCP spokesperson will be referred to from here on as Ellis].

Ellis claims he stopped doing cybercrime for TeamPCP in March 2026 — just before the attacks that compromised LiteLLM — and that at least one other individual has taken over the group’s leadership since then. Ellis shared that a year earlier he had just completed the latest in a series of detox and sobriety programs, and was two months sober when he reconnected with some old friends from the malware development scene.

“One year ago I needed help monetizing some [GitHub credentials], I was two months sober and needed a distraction and something to keep busy as well as people to speak to,” Ellis said. “I had largely disconnected from my old circle, they had become very toxic and I needed to get away from the substances. Previously I had done some mass exploitation campaigns and grew up doing [malware development] and [capture the flag] contests. There were some friends who were also vending but had stopped a while, and one of them introduced me to some chats where I posted access for sale.”

Prior to that, Ellis said, he was homeless and hopping between “some very unstable places.”

“Blackhatting is fun,” he said. “There are actual rewards and incentives to learn and you grow with your team. Without qualifications, no employer will even take the time to hear you out.”

Ellis claims he’s earned a grand total of about $20,000 for his activities with TeamPCP, and that it was never about the money or fame for him. Asked whether his experiences with TeamPCP might prepare him for gainful employment in a legitimate IT job, Ellis said he doubted it.

“I am nowhere close to a skill level where I am comfortable, and this would take maybe half a decade of further experience,” he said. “I no longer have to choose between rent and food for that I’m grateful and so are the team members.”

Ellis expressed no remorse over his cybercrime activities, and said he was grateful for the friendships and relationships built throughout his engagement with TeamPCP. The young hacker also seemed resigned to his fate, and told KrebsOnSecurity that he’ll accept the consequences if he’s ever arrested.

“If I’ve already been found out then its out of my control, I’ll make peace with that,” he said. “Honestly, I think someone like me needs a lot of help that prison just can’t provide. If I had the funds to study different parts of the field and closer guidance, this would have turned out differently. But that’s a pipe dream and we both know this.”

It is clear from reading Ellis’s posts to the group’s Matrix server chats that his struggles with sobriety are ongoing. On Thursday, June 25, Ellis told @kernelstub he was about to “trip” with his “homie.”

“What kind,” @kernelstub inquired.

“Ketty and some DMT,” Ellis replied, referring to the dissociative anesthetic ketamine and dimethyltryptamine (DMT), a powerful psychedelic compound that is found naturally in some plants but is also synthetically produced in underground lab environments. “There’s a little 2cb so we might throw that in the mix,” he continued, referring to another psychedelic compound by its chemical shorthand.

Roughly two weeks before his arrest, Ellis told KrebsOnSecurity he was ready to leave his life of crime behind and was prepared to turn himself in, but that in the meantime he was making plans to tie up loose ends.

Less than 24 hours later, the TeamPCP leader posted an image on Telegram showing a yellowish powdered substance in a baggie and on a scale, possibly synthetic DMT. The image shows the powder being weighed next to a series of small vape cartridges, two of which are open on the table in front of the photographer.

An image posted by the TeamPCP leader to Telegram, advertising his acquisition of some type of psychoactive substance, most likely a synthetic version of the powerful hallucinogen known as DMT.

The two defendants were arrested Wednesday morning. The AFP said the men face a combined 14 cybercrime offenses and are scheduled to appear in Perth Magistrates Court today.

Charlie Eriksen is a security researcher at Aikido Security who has closely followed TeamPCP’s cybercrime campaigns. Eriksen said TeamPCP are a good example of a new kind of threat actor that does not fit neatly into the usual categories.

“They are not a state actor, not quite organized cybercrime, and not purely ideological,” he said. “Their motivations seem to mix money, disruption, attention, and ideology.”

Eriksen said that historically there has always been a meaningful gap between reading about an attack technique and being able to reliably turn it into an operational campaign, but that large language models (LLMs) and artificial intelligence increasingly are helping threat actors to bypass that knowledge gap.

“You had to understand the research, adapt the code, troubleshoot it, build infrastructure around it, and then repeat that process across different targets,” he said. “LLMs have compressed that gap significantly.”

According to Eriksen, this creates an environment where threat actors suddenly have the ability to operate at significant scale without having developed the operational discipline that traditionally accompanies that level of capability. Put another way, it sets the stage for cybercriminals who are capable enough to cause significant damage, but not necessarily careful enough to understand or care about the consequences.

“They can be noisy, they can make mistakes,” he said. “They can leave evidence everywhere. They can take risks that a professional criminal group or intelligence service would consider completely unacceptable. But that does not necessarily make them less dangerous. In some ways, it can make them more dangerous.”

In a recent blog post, Eriksen called TeamPCP’s Shai-Hulud worm the “best thing to happen to supply chain security,” because it forced GitHub and other public coding platforms to erect new security safeguards.

In direct response to TeamPCP’s broad success at pushing poisoned versions of popular software packages, GitHub in late July introduced a three-day “cooldown” mechanism for Dependabot, the platform’s tool for auto-fetching newly shipped updates for any package dependencies. Cooldown periods are designed to help buy time for security tools and package maintainers to identify and remove any compromised versions. Other coding ecosystems like Python and various JavaScript platforms also added support for cooldown periods this year amid growing calls from security experts about the need for more widespread adoption of the safety feature.

Eriksen said TeamPCP’s legacy is that they achieved in the span of a few months what the supply chain security community has been unable to do for years.

“They managed to wake up Microsoft to the fact that they had become negligent in terms of security,” Eriksen said. “By compromising GitHub and stealing their source code, they humiliated Microsoft into action, making them finally act on what we had been asking them to do and take seriously for a while now.”

Update, 10:08 a.m. ET: A story this morning from ABC News in Australia confirms Ruben Ian Thomson of Cottesloe was one of the two arrested. The 23-year-old suspect thought to be @pcpcasper, Michael Gaebler, also was arrested in Perth. ABC News reports that Thomson was denied bail (Mr. Gaebler’s attorney reportedly did not request bail for his client), and that both men will be held in custody until their next court appearance on September 18.

Top

Launching Route 53 Files

Post by Colin Percival via Daemonic Dispatches »

I'm excited to announce Route 53 Files, a new file system that seamlessly connects any AWS compute resource with Amazon's highest-availability database.

Top

Strom für ein elektrisches Deutschland

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

Deutschland verbraucht heute sehr viel Energie, aber nur ein kleiner Teil davon ist Strom. Das ist kein Argument gegen Elektrifizierung, sondern ihr größter Vorteil.

Ein Liter Diesel enthält ungefähr 10 kWh Energie. Ein Dieselauto mit 6 l/100 km verbrennt also 60 kWh/100 km. Am Rad kommen grob 20 kWh/100 km an; der Rest wird Wärme. Ein Elektroauto braucht für die gleiche Strecke meist weniger als diese 20 kWh aus der Batterie.

Das Prinzip gilt nicht nur für Autos. Fossile Energie wird meist verbrannt, um Wärme oder Bewegung zu erzeugen. Elektrische Motoren, Wärmepumpen und elektrische Prozesse umgehen einen großen Teil dieser Verluste.

Ich habe deshalb eine grobe, aber konkrete Rechnung gebaut: historische deutsche Viertelstundenwerte aus 2024, auf ein elektrifiziertes Deutschland hochskaliert, mit Wind, Solar, Vier-Stunden-Batterien und Gaskraftwerken als letzter Reserve. Der Quellcode liegt auf GitHub: isotopp/strommodell .

Die Zielgröße: rund 1.100 TWh Strom

Die AG Energiebilanzen hat vollständige Anwendungsbilanzen derzeit bis 2024. Das detaillierte Energieflussbild 2024 weist aus:

  • 10.542 PJ (2.928 TWh) Primärenergieverbrauch;
  • 8.095 PJ (2.249 TWh) Endenergieverbrauch;
  • 1.664 PJ (462 TWh) davon Strom;
  • 6.431 PJ (1.786 TWh) übrige Endenergie.

Für die Frage nach einem elektrischen Deutschland ist der Endenergieverbrauch die sinnvolle Basis. Der Primärenergieverbrauch enthält die Verluste heutiger Kohle- und Gaskraftwerke bereits. Wer ihn nimmt und dann fossile Energie noch einmal pauschal durch drei teilt, rechnet einen Teil der Verbesserung doppelt.

Die absichtlich grobe Rechnung lautet:

$$ E_\text{elektrisch} = E_\text{Strom} + \frac{E_\text{übrige Endenergie}}{3} $$

also:

$$ 1.664\ \text{PJ} + \frac{6.431\ \text{PJ}}{3} = 3.807\ \text{PJ}. $$

Das sind 1.058 TWh im Jahr. Mit Netz- und Speicherverlusten und der unvermeidlichen Ungenauigkeit einer solchen Faustformel runde ich auf 1.100 TWh/a.

Zum Vergleich: Der heutige Stromverbrauch einschließlich Netzverlusten betrug 2024 nach SMARD-Daten der Bundesnetzagentur 465,5 TWh. Die Zielgröße ist damit das 2,36fache der heutigen Stromarbeit.

Reale Viertelstunden statt Jahresmittel

1.100 TWh im Jahr entsprechen einer mittleren Leistung von 126 GW:

$$ \frac{1.100 TWh}{8.760 h} = 126 GW. $$

Ein Stromsystem muß aber in jeder Viertelstunde funktionieren. Deshalb verwende ich die öffentliche deutsche 2024er Zeitreihe von Fraunhofer Energy-Charts : Last, Solar, Wind an Land und Wind auf See, jeweils in 15-Minuten-Schritten.

Die reale Lastreihe hat 465,503 TWh. Sie wird proportional auf 1.100 TWh skaliert. Ihre Form bleibt dabei unverändert. Das ergibt eine Spitzenlast im konkreten 2024er Profil von 179,0 GW.

Für jede Viertelstunde werden PV, Wind an Land und Wind auf See getrennt skaliert:

$$ P_\text{Szenario}(t) = P_\text{beobachtet}(t) \cdot \frac{K_\text{Szenario}}{K_\text{Referenz}}. $$

Die Referenzleistung ist für 2024 der Mittelwert aus dem Anlagenbestand Ende 2023 und Ende 2024. Das ist bei starkem PV-Zubau besser als blind mit dem Jahresendwert zu rechnen.

Dann bleibt die Residuallast:

$$ R(t) = D(t) - S(t) - W_\text{an Land}(t) - W_\text{auf See}(t). $$

Bei Überschuß lädt die Batterie, danach wird abgeregelt. Bei Defizit entlädt sie, danach liefert Gas. Es gibt im Modell keine Importe und keine ungedeckte Last.

Batterie für Stunden, Gas für Flauten

Die Batterie hat in den Szenarien vier Stunden Energieinhalt: 100 GW Batterie-Leistung bedeuten 400 GWh Energie. Sie lädt und entlädt jeweils mit 90 % Wirkungsgrad.

Das verschiebt Solarstrom vom Mittag in den Abend und glättet kurze Windschwankungen. Es ist aber kein saisonaler Speicher. Wenn bei hoher Last Wind und Sonne über viele Stunden praktisch ausfallen, ist eine Vier-Stunden-Batterie leer.

Gaskraftwerke decken ausschließlich den Rest:

$$ G(t) = \max(0, R(t) - B_\text{entladen}(t)). $$

Die entscheidenden Ausgaben sind daher getrennt:

  • Gasarbeit: Wie viele TWh müssen im Jahr aus Gas kommen?
  • Gasleistung: Wie viele GW müssen im schlimmsten Viertelstundenschritt bereitstehen?

Die zweite Zahl bestimmt die Größe der Reserveflotte, nicht die erste.

Vier Szenarien

Szenario 0 benutzt den Bestand Ende 2024: 99,3 GW PV, 63,53 GW Wind an Land, 9,215 GW Wind auf See und 12,67 GW / 18,65 GWh Batteriespeicher. Die Batterie hat damit heute nur etwa 1,5 Stunden Dauer.

Szenario PV Wind an Land Wind auf See Batterie
0: Iststand 2024 99,3 GW 63,53 GW 9,215 GW 12,67 GW / 18,65 GWh
A: knapp 300 GW 250 GW 70 GW 50 GW / 200 GWh
B: Referenz 400 GW 300 GW 80 GW 100 GW / 400 GWh
C: viel Überschuß 500 GW 350 GW 100 GW 150 GW / 600 GWh

Szenario 0 ist ausdrücklich nicht die heutige deutsche Strombilanz. Es setzt den heutigen Wind-, PV- und Batteriepark auf die Last eines elektrifizierten Deutschlands und fragt: Was bliebe dann für Gas übrig?

Ergebnis: Gasarbeit fällt, Gasleistung fast nicht

Der Modelllauf für 2024 ergibt:

Szenario Gasleistung Gasarbeit Stunden mit Gas Abregelung
0: Iststand 2024 163,3 GW 896,0 TWh 8.784 h 0,0 TWh
A: knapp 156,7 GW 319,4 TWh 5.548 h 56,9 TWh
B: Referenz 156,6 GW 214,2 TWh 3.545 h 127,5 TWh
C: viel Überschuß 156,5 GW 133,4 TWh 2.086 h 253,3 TWh

Das Resultat ist klar.

Von Szenario 0 zu C sinkt die Gasarbeit von 896 auf 133 TWh, um 85 Prozent. Die Zahl der Viertelstunden mit Gaseinsatz sinkt von praktisch dem ganzen Jahr auf 2.086 Stunden, also um 76 Prozent. Das sind weiterhin nicht „wenige Tage“: Selbst im großen Szenario C läuft Gas noch in rund einem Viertel des Jahres, wenn auch oft nur mit kleiner Leistung.

Die Gasleistung sinkt dagegen kaum: von 163,3 auf 156,5 GW. Am 6. November 2024 um 16:30 UTC liegt die hochskalierte Last bei 157,2 GW. Wind und Solar liefern im Szenario C zusammen nur 0,66 GW. Die Batterie ist zu diesem Zeitpunkt leer. Genau für diese Lage muß die Gasflotte da sein.

Mehr Wind, Solar und Batterie machen Gas deshalb von der dominanten Energiequelle zur Rest- und Dunkelflautenenergie. Sie machen die Gaskraftwerke aber nicht überflüssig.

Das wahrscheinliche Ergebnis ist eine große Flotte schneller Gaskraftwerke als Peaker auf Abruf. Sie verkauft wenig Energie, muß aber im kritischen Moment vollständig verfügbar sein. Das bezahlt man nicht sinnvoll allein über verkaufte MWh, sondern über eine Vergütung für vorgehaltene gesicherte Leistung.

Die Abregelung in B und C ist kein Fehler. Sie ist der Preis dafür, daß Wind und Solar auch in schlechten Stunden noch ausreichend oft viel Energie liefern. Zusätzliche Speicher oder flexible Verbraucher können einen Teil dieser Überschüsse nutzen; sie ändern aber nicht die einfache Tatsache, daß eine lange, dunkle und windarme Winterphase gesicherte Leistung braucht.

Der vollständige Rechenweg ist reproduzierbar:

uv run strommodell run scenarios/2024.yaml --output results/2024
uv run strommodell report results/2024

Grenzen des Modells

Das ist eine Daumenschätzung mit einer realen Jahreszeitreihe, keine Ausbauplanung.

  • Es wird absichtlich nur 2024 gerechnet. Für die Größenordnung reicht das; für eine verbindliche Reserveplanung müßte man viele Wetterjahre und besonders schlechte Dunkelflauten untersuchen.
  • Die Lastform von 2024 wird nur proportional hochskaliert. Wärmepumpen, Elektroautos, Industrie und Elektrolyse bekommen noch keine eigenen, steuerbaren Profile.
  • Die Skalierung verwendet beobachtete Einspeisung. Darin stecken heutige Abregelungen und Netzrestriktionen; ein künftiger, viel größerer Anlagenpark hätte nicht exakt dasselbe Profil pro installiertem GW.
  • Der Anlagenbestand wird nur über den Mittelwert aus Jahresende 2023 und 2024 angenähert. Eine zeitgenaue Zubaukurve wäre besser.
  • Wasserkraft, Biomasse, Fernwärme, Pumpspeicher, europäischer Stromhandel, Netze und Lastmanagement fehlen. Im Modell ist die Welt absichtlich härter: Wind, Sonne, Batterie und danach Gas.
  • Der Batteriespeicher startet halb voll und darf mit einem anderen Ladezustand enden. Das ist bei den Jahreswerten klein, aber für einen vollständigen Systemoptimierer müßte der Jahresübergang zyklisch behandelt werden.
  • Wasserstoff wird nicht modelliert. Ob man für selten laufende Peaker Wasserstoff wirtschaftlich erzeugen und speichern kann, ist eine eigene Frage. Elektrolyseure skalieren ökonomisch bei sehr geringer Auslastung sehr schlecht; das ist kein Problem, das diese einfache Rechnung lösen kann.

Quellen

Top

FreeBSD Foundation Intern Sourojeet Adhikari on Bringing ROCm to FreeBSD

Post by FreeBSD Foundation via FreeBSD Foundation »

For University of Waterloo student Sourojeet Adhikari, an early interest in understanding how computers work eventually grew into a summer spent helping bring AMD’s ROCm GPU computing platform to FreeBSD.

Sourojeet is studying Mathematical Physics and entering the third year of his degree. This summer, he joined the FreeBSD Foundation as an intern, working on a project that combines several of his interests: systems programming, GPU computing, and open source.

We spoke with Sourojeet about his path into open source, the challenges of bringing ROCm to FreeBSD, what he learned from debugging difficult problems, and why students shouldn’t be intimidated by contributing to a large open source project.

An Early Introduction to Open Source

Sourojeet’s interest in open source began when he built his first computer. Without a Windows installation disc or drive available, he turned to what he could find: an Ubuntu 12.04 installation CD sitting on a shelf in his basement.

“I installed Ubuntu and spent days figuring out how to use it,” he said. “From then on, I more or less ran Linux on almost every computer or server I’ve ever owned.”

FreeBSD was also part of his early exposure to open source. One of his parents ran a FreeBSD firewall and another FreeBSD server for a time. The firewall is still running today.

What kept Sourojeet interested was the freedom to explore.

“Open source software generally let me play around with and look deep into the internals of software, modify it, and try to break it,” he said. “That’s why I loved it so much.”

His interest in systems programming came from another question: How can I make my programs faster?

Understanding performance meant understanding what a system was doing under the hood. That curiosity led him deeper into systems programming and, eventually, GPU programming.

Bringing ROCm to FreeBSD

Sourojeet found the FreeBSD Foundation internship through WaterlooWorks, the University of Waterloo’s co-op job board.

His internship project focused on bringing AMD’s ROCm platform to FreeBSD.

Modern computers rely on different components designed for different kinds of work. CPUs are general-purpose processors built to handle a broad range of tasks, while GPUs are highly effective at performing certain kinds of mathematical operations in parallel.

That makes GPUs useful for much more than graphics. Workloads including machine learning, fluid simulations, and data processing can benefit significantly from GPU acceleration.

Platforms such as NVIDIA’s CUDA and AMD’s ROCm give developers the tools needed to take advantage of that computing power. However, these ecosystems have primarily focused on Windows and Linux, with limited native support on FreeBSD.

Sourojeet’s goal was to help change that.

His work involved patching AMD’s LLVM fork, making changes to ROCm’s GPU runtimes, and working to integrate the driver into drm-kmod. Some of the LLVM changes have already been upstreamed, while other pieces of the work are still being merged and developed.

“Slowly but surely, we’ll soon be able to natively run ROCm code on FreeBSD,” he said.

Finding the Bugs That Don’t Want to Be Found

For Sourojeet, one of the most rewarding parts of the internship was also one of the most difficult: debugging.

One issue took several hours to track down and came from a subtle difference between mutable and constant data.

The problem involved LinuxKPI, a compatibility layer in the FreeBSD kernel that helps translate or provide Linux kernel interfaces needed by certain drivers. A Linux function called class_register had changed in newer Linux versions to accept a const structure. The AMD kernel code Sourojeet was porting expected that newer behavior, while the corresponding FreeBSD LinuxKPI implementation still expected a mutable structure.

The mismatch caused a kernel panic.

“It took me maybe three or four hours to figure that one out,” Sourojeet said.

Rather than being discouraged by those problems, he found them to be some of the most enjoyable parts of the work.

“The most fun part was generally debugging the hard-to-find bugs and issues in the code.”

Working Through the Tedious Parts

Not every challenge involved a single difficult bug. Much of the porting work required dealing with Linux-specific macros and functions that did not yet exist on FreeBSD.

Sourojeet first had to determine whether a missing function or code path was actually necessary. Some functionality, such as code related to suspend or sleep behavior, was not required for the initial work and could be temporarily disabled. In other cases, a missing function was too important to remove and needed a temporary stub so development could continue.

The individual changes were not always technically complex. The challenge was often the amount of careful, repetitive work required to move the project forward.

“The actual changes you needed to make to get the driver running weren’t hard or insane,” he said. “Rather, they were just mentally grueling and very tedious.”

That experience also reinforced something Sourojeet wants other potential contributors to understand: meaningful open source work does not always require being an expert.

“If you’re reading this and have some ability to read and write C code, you could probably help out with almost everything.”

Connecting With the Community at BSDCan

During the summer, Sourojeet also attended BSDCan, giving him an opportunity to connect with members of the BSD community in person.

One talk that stood out was Minsoo Choo’s presentation on heterogeneous scheduling on FreeBSD. Sourojeet was particularly interested in the discussion around scheduling costs and performance across different CPU cores.

But some of his favorite parts of BSDCan happened outside the formal sessions.

“The hallway track was a lot more interesting from what I remember,” he said.

Those conversations covered topics ranging from software testing and GPU initialization to heterogeneous memory management on FreeBSD. The latter is especially relevant to where Sourojeet’s own work may go next.

Open Source Is More Approachable Than It Looks

After spending the summer working directly within the FreeBSD ecosystem, Sourojeet came away with an even stronger belief that contributing to open source is more accessible than many new developers assume.

“This internship reinforced my belief that open source is not particularly scary or hard to contribute to,” he said. “It just needs some time and energy.”

His advice to students and first-time contributors is simple: don’t wait until you think you know everything.

“Don’t wait until you think you’re perfect, or the best at coding,” he said. “A lot of the work is tedious debugging, writing docs, or even implementing basic functions.”

Large codebases can look intimidating from the outside, but Sourojeet encourages new contributors to focus on the problem in front of them rather than the size of the entire project.

“It’s just code—maybe a lot of code—but I have faith that anyone who puts in some time and energy can come out with some meaningful changes.”

What’s Next?

Sourojeet expects his work with ROCm and FreeBSD to continue beyond the internship.

The project is already approaching an important milestone: running a simple vector addition workload. The driver can link, load, and run, although there are still issues to resolve in userspace.

From there, Sourojeet expects to spend more time exploring FreeBSD’s memory management systems, particularly the work required to support Heterogeneous Memory Management (HMM).

As for his longer-term career plans, he is keeping his options open.

His ideal path may involve computational fluid dynamics—or, as he jokes, making enough money to eventually retire and hire interns to tackle difficult problems like HMM on FreeBSD. But his experience this summer has also opened another possibility: working on GPU toolchains.

Whatever direction he takes, the internship has given him practical experience solving problems deep within a complex open source system, and a reason to keep contributing.

Sourojeet also credits the people who supported him throughout the project, including Ed Maste, Olivier Certner, Mark Johnson, Jean-Sébastien Pédron, Bjoern Zeeb, Adrian Chadd, Minsoo Choo, and Devin Teske.

His summer is also a reminder for students considering their first open source contribution: you don’t need to understand an entire operating system before you can make an impact. Start with a problem, spend the time to understand it, and contribute what you can.

The FreeBSD Foundation would also like to thank Sourojeet for his contributions throughout the semester. We appreciate the curiosity, persistence, and technical effort he brought to the internship, as well as his willingness to tackle challenging problems and continue pushing ROCm support on FreeBSD forward. 

The post FreeBSD Foundation Intern Sourojeet Adhikari on Bringing ROCm to FreeBSD first appeared on FreeBSD Foundation.

Top

Valuable News – 2026/08/24

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD 14.5-BETA3 Now Available.
https://lists.freebsd.org/archives/freebsd-stable/2026-August/004283.html

FreeBSD Git Weekly: 2026-08-10 to 2026-08-16.
https://freebsd-git-weekly.tarsnap.net/2026-08-10.html

FreeBSD Git Weekly: 2026-08-17 to 2026-08-23.
https://freebsd-git-weekly.tarsnap.net/2026-08-17.html

In Praise of BSD License.
https://reddit.com/r/freebsd/comments/1vqsb7b/praise_bsd_license/

Migrating from Codeberg Pages to OpenBSD VPS.
https://nemin.hu/vps/

FreeBSD 15 on IBM PowerVM LPAR with Matching pkg(8) Repo for ppc64le Arch.
https://librepower.org/platforms/freebsd/

BSDun Allows Running FreeBSD ELF Executables on Linux.
https://gitlab.com/megastallman/bsdun

NetBSD 11 Lands with RISC-V Support and Lightning Fast VM Boots.
https://theregister.com/os-platforms/2026/08/20/netbsd-11-risc-v/5289713

Say Hello to FreeBSD on PowerVM.
https://librepower.substack.com/p/say-hello-to-freebsd-on-powervm

Specially Crafted NTFS Filesystem Image Allows Root Access on Linux.
https://phoronix.com/news/NTFS3-Vulnerability-For-Root

FreeBSD Foundation Intern Jim Huang Chen on Raspberry Pi Support and Open Source.
https://freebsdfoundation.org/blog/freebsd-foundation-intern-jim-huang-chen-on-raspberry-pi-support-and-open-source/

BSD Weekly – Issue 292.
https://bsdweekly.com/issues/292

NAS Upgrade – Initial Planning.
https://vulcanridr.mataroa.blog/blog/nas-upgrade-initial-planning/

Setup Your Own IPFS Private Network.
https://dataswamp.org/~solene/2026-08-21-your-own-ipfs-network.html

Why Some Satellites Use NetBSD?
https://machaddr.substack.com/p/why-some-satellites-use-netbsd

How to Test MediaTek MT7921 on FreeBSD.
https://lists.freebsd.org/archives/freebsd-wireless/2026-July/004372.html

Analyze FreeBSD Build Process with srcpv(8) Tool.
https://linkedin.com/posts/dteske_freebsd-ugcPost-7494829603732193280-s2fD/

Experimental Roblox Compatibility Runtime for Linux and FreeBSD.
https://github.com/komaruworld/mocktail

UNIX/Audio/Video

Running Large Language Model with Ollama on FreeBSD.
https://youtube.com/watch?v=6MRvKcWse7c

Responsible Replication with Zelta on ZFS.
https://youtube.com/watch?v=G3weooQqcXw

BSD Now 677: Butler at Your Service.
https://www.bsdnow.tv/677

Hardware

AMD Chipset Cards Adding PCIe/SATA Lanes Emerge in China.
https://hwcooling.net/en/amd-chipset-cards-adding-pcie-sata-lanes-emerge-in-china/

This is Not Computer for You: Appreciation of Struggle.
https://stevengharms.com/posts/2026-08-21-this-is-not-the-computer-for-you-an-appreciation-of-struggle/

Life

Need for Speed.
https://filfre.net/2026/08/a-need-for-speed/

IP Problem Hiding in AI Generated Code.
https://medium.com/@markus_brinsa/the-ip-problem-hiding-in-ai-generated-code-88b30071e0a2

Other

Nintendo Wipes Out 400+ Switch Emulator Repos in Single Day GitHub Sweep.
https://torrentfreak.com/nintendo-wipes-out-400-switch-emulator-repos-in-single-day-github-sweep/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Building server-key-injection by specialization

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

server-key-injection is an SSH certificate issuer. A user connects with ssh -A, authenticates, and receives a new, short-lived keypair and certificate in their existing ssh-agent. Production hosts trust the user CA and make their authorization decision locally.

It is also an experiment in software development with a language model. No line of code outside /developer was written by a human. The source tree, tests, configuration, installer, and documentation were generated by an LLM in a code-generation harness. I wrote, reviewed, and repeatedly revised the requirements and their specialised derivatives.

That was the whole point of the exercise: The project is as much a proof of concept for SSH key injection as it is a test of a development process.

More precisely, I wanted to answer three questions:

  1. Can I develop a systematic vibecoding workflow, rather than merely have a sequence of lucky chat sessions?
  2. What code quality does that workflow actually produce?
  3. What teachable rules and background explain why the workflow functions when it does?

The answer to the first question is the specialization workflow below. The second is not answered by aesthetic preference alone: the code must be readable, the test suite must establish useful behavioural boundaries, and the result must survive implementation, security, and refactoring reviews. The third is the useful part, because a workflow which cannot be explained cannot be reliably repeated or improved.

The specialization workflow treats the model as a fallible implementation resource needing oversight, and tries to work from a lose discussion about a piece of software to hard requirements with mechanistic verification, with an intermediate step of a disposable “tracer” implementation verifying viability of the concept and tools. Using git and fresh contexts we have recovery points if at one step the entire codebase goes left. We anticipate that several feature-generation steps will accumulate cruft and pause in regular intervals for cleanup steps.

Requirements first

The central artifact is developer/architecture.md . It attempts to describe the system before code exists:

  • why the issuer exists and how the credential flow works;
  • the CLI surface and the user experience;
  • components, trust boundaries, and the production-host decision path;
  • the replaceable identity-provider boundary; and
  • the required security properties.

This is not a broad technical vision document. It is a deliberately concrete description of the desired behaviour. It says which key is long-lived, which key is ephemeral, where group membership is captured, what a target host may trust, what it must decide locally, and what must never be logged or persisted.

Writing this resembles the work of a product owner or technical program manager: progressively remove ambiguity until a later implementer has little room to choose an incompatible solution. Here the implementer happens to be a model.

The effect is incremental removal of degrees of freedom. I call the result a specialization workflow. A broad goal such as “issue short-lived SSH keys” does not constrain much. An architecture with exact principals, validity periods, trust boundaries, filesystem rules, and error behaviour constrains a great deal more. A user story constrains it further. A ticket should leave a small, testable change.

The disposable tracer

The first epic was intentionally throwaway: developer/2026-08-20-dummy-issuer-and-agent-injection-tracer . It proved the hard and unfamiliar part first: AsyncSSH can accept an ssh -A connection, reach the forwarded agent, inject a short-lived, non-functional certificate identity, and make it visible with ssh-add -l.

The test CA was disposable. No production host trusted it. The credential had a one-hour, test-only lifetime. There was no persistent CA state, no user database, no TOTP, no production authorization, and no useful login. It was a tracer: evidence that the wire-level construction was possible.

That tracer was later scrapped. Its value was not the code; it was the reduction of uncertainty: We proved that the protocol could do what we wanted, that the libraries chosen implement the functionality and that the problem in general was solvable with a model.

Once that we done we could (and did) scrap that code.

From epics to code

The architecture defines epics. Each one became a dated directory under developer/, such as developer/2026-08-21-persistent-ca-and-ordinary-certificate-issuance . That directory first received structured user-stories.md. Those stories were then developed into ordered, actionable tickets.md. Only then were tickets turned into tests and code.

The repository’s AGENTS.md contains the workflow verbatim:

## Specialization workflow

We never generate code ad hoc unless the user specifically requests it. A
direct request is a scoped bypass for debugging, an ad-hoc fix, or an
experiment; Git provides the recovery path for that work.

Otherwise, follow this directory-based workflow for the epic currently being
worked on. Do not modify artifacts belonging to another independent epic or
user story.

1. **User-story step.** Create a directory named
 `developer/<YYYY-MM-DD>-<epic-slug>/`. Put the epic's structured user stories
 in `user-stories.md` and any relevant reviews in that directory, for example
 `security-review.md` or `refactoring-review.md`.
2. **Ticket step.** Commit the current epic's relevant user-story or review
 file before this step. Develop it into actionable tickets in `tickets.md` in
 the same directory, ordered for implementation. Use the TDD skill while
 developing tickets.
3. **Code-generation step.** Commit that epic's `tickets.md` before beginning
 this step. Generate code from the tickets using the TDD skill. When a ticket
 is complete, commit it using the git-commit skill; only then proceed to the
 next ticket.

Committing between stages is useful. It makes the upstream decision durable, visible, and independently reviewable. It also gives the next context a stable starting point instead of a mixture of mutable prose, provisional tickets, and unfinished code.

The stages do not have equal safety properties. Tests gate the last stage mechanically. The architecture, stories, and tickets are primarily manual deliverables. That is where active, sometimes aggressive editing matters. Generating plausible but wrong tickets is a common symptom of an ambiguous architecture or user-story file, not a reason to patch around the problem in code. Delete the bad ticket file, fix the upstream requirement, start a fresh context, and regenerate it.

A harness, not a chat window

This does not work as a conversation pasted into a generic chat interface. The model needs a code-generation harness such as Codex or OpenCode. The harness gives it a Git repository, a shell, tests, and files it can create and read.

Those files are the project’s long-term memory. The model context is working memory: useful for the task in flight, but limited and disposable. The repository contains the durable decisions needed when the working context has ended.

The harness also enables progressive discovery. A model should not be given the entire repository, which is how one gets a large and contradictory context. It should discover the relevant context by reading index files and task-local guidance. A good index is a table or list of filenames with two or three sentences explaining what each file contains and when it should be read. README.md and AGENTS.md are particularly useful names because they make the index discoverable before the model has learned the project layout.

There are two complementary forms of context building:

  • Automatic context building is progressive discovery: the model reads the repository guides, follows their references, and loads what the current task needs.
  • Manual context building is a human deliberately asking the model to read certain files, discussing the result, and only then authorizing a modification.

The second is why it is better to develop architecture.md in dialogue with the model than to write it alone and ask for a later implementation. The discussion preloads the relevant context and directs attention to the constraints which matter.

Context management has a cost. Resetting a tainted context discards useful implicit rationale together with the contradiction. The answer is not to keep an ever larger transcript. It is to promote the rationale which must survive into concise, discoverable files, and to make a fresh context read those files before it acts. Context management for long-horizon software agents is an active engineering problem, not merely a prompt-writing detail; see Context as a Tool .

Architecture Decision Records are sometimes useful for recording a narrow, local choice. Their usefulness is limited, though: They help to keep the model on track, and prevent it from quietly reversing an earlier decision. They make automatic reconstruction of necessary context and decision history possible when starting a fresh new context.

architecture.md is different: it is the durable description of how we build the system – and what we build in the first place: constraints, and direction.

Skills and TDD

The project keeps skill guides in .agents/skills. They are part of the repository, both as documentation and so every contributor uses the same instructions. The essential one here is the Matt Pocock TDD skill.

It changes the code-generation step from “write something that sounds right” to a red-green-refactor loop:

  1. Decide the behavioural boundary.
  2. Write a test that demonstrates the missing behaviour.
  3. Lock the test in place.
  4. Write the smallest public implementation which makes it pass.
  5. Refactor only while the test stays green.

This does not prove the design correct. It does make a substantial class of regressions mechanically visible, and it forces the model to identify an observable contract before it reaches for implementation detail. The same repository guidance requires formatting, linting, type checking, and the test suite before handoff. This is also the core of the code-quality evaluation: manual review decides whether the requirements are the right ones, while the mechanical gates decide whether later changes continue to meet them.

The tests must include the boundaries which are too important to model only with units. For this project that means targeted integration evidence around the forwarded agent, the real sshd certificate path, target-host policy, file ownership, and the Rocky/SELinux environment. A unit test can establish that an adapter was called; it cannot prove that a production OpenSSH server accepts the right certificate and rejects the wrong one. Targeted non-unit tests and repeatable VM checks are where those claims belong.

Model choice and cost

Different stages benefit from different models. For the architecture and user-story stages I used ChatGPT-5.6/Sol at medium or high reasoning effort. The output was useful, but needed several interactive passes and substantial editing. That is the right place to spend attention, because errors multiply as requirements specialize downstream.

Ticket generation can use ChatGPT-5.6/Terra at medium or high reasoning. With good upstream documents, tickets usually need reading more than editing. Code generation used ChatGPT/Luna at xhigh: fast, very capable at the constrained task, and remarkably cheap.

The whole project consumed part of a weekly allowance from a monthly USD 20 ChatGPT subscription. Roughly USD 2.50 was attributable to this project. Around two thirds of that was spent on the larger models for architecture and stories; almost none was spent on generation of the code itself.

That ratio makes sense. Cheap code generation is valuable only after the problem is narrowly enough stated that cheap code generation has few damaging choices available.

Context is an engineering concern

Models produce worse work when the context is too large, when required facts are hidden on disk, or when it contains contradictory statements: “do this” next to “never do this”. I call the last condition tainted context.

Do not argue indefinitely with tainted context. Drop it. Start a new task or thread, preload the relevant files, ask for discussion rather than file generation, and issue the implementation request only once the model has seen the decisions which constrain it. This is not ritual; it is input hygiene.

The same discipline applies after several epics. Cruft accumulates even when tests pass. Every three to five epics, run a refactoring review, security review, test-quality review, or another appropriately scoped review. Select which findings to accept, cut scope deliberately, turn accepted findings into tickets, and run the same specialization workflow again.

One particularly useful form is an Adversarial Review. Give it to a fresh model context, or to a human who did not generate the change, and ask it to find evidence that the implementation, tickets, or tests are wrong. Do not ask the original generation context to approve its own reasoning; it is biased toward its framing and its omissions.

Test review deserves the same adversarial treatment. Read the tests; then review the tests, separately from the implementation. Ask specifically for tests which do nothing: tests with no meaningful assertion, tests which only prove that a called function received its parameters, tests which only exercise a mock, or tests which merely duplicate an implementation detail. They add maintenance cost and can be dropped.

Also ask which tests would survive unchanged under a plausible internal refactor. Those probably test a useful observable behaviour at the right seam. The aim is not a larger test count; it is tests which make the intended public contract harder to break. A focused request works: “Review these tests. Good looks like tests which observe a behavioural boundary, do not merely inspect parameter passing, and survive an internal refactor unchanged.” “Review this” is too vague for a model to apply a useful standard.

I am satisfied with the result. The code is not shaped as I would have written it by hand, but it is readable, fairly clean, and surrounded by tests. It has also been through refactoring and security review, and their accepted findings were fed back through the same story-ticket-code loop. A test review and the final epic are pending.

That is not a claim that the result is perfect. It is evidence that the workflow can produce code whose quality is inspectable, testable, and subject to ordinary engineering correction.

The teachable rules are equally ordinary: specialize requirements to remove degrees of freedom, make durable decisions discoverable on disk, use mechanical gates wherever the deliverable permits them, and discard tainted context instead of building on it.

This is not autonomous software engineering, and that is intentional. Models are probably not good at autonomous engineering yet, especially not for security-relevant work. During this project I had to limit scope and nip baroque architectures in the bud several times; models are particularly bad at YAGNI. A human still chooses the goal, the threat model, the acceptable risk, and the point at which a clever construction is simply too much construction.

We also have to read the tests and review the tests. An adversarial test review is not optional decoration around green CI. It asks whether tests assert useful behaviour, whether targeted non-unit checks cover the real boundaries, and whether apparently comprehensive tests are only checking that mocks received parameters. Focus the review: do not tell a model “Review this.” Tell it what good looks like, including tests which survive refactoring unchanged and tests which exercise the right seam. The process is not “ask an LLM to write a program.” It is to progressively remove degrees of freedom until a model can safely do a small, well-defined piece of engineering.

Sources

Top

Reading the ski SSH certificate issuer

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

ski is a small SSH certificate issuer. A user logs in to it, completes a password-and-TOTP exchange, and receives a new Ed25519 keypair plus a signed OpenSSH user certificate in their forwarded ssh-agent.

The code does not implement its own crypto or mechanism. It uses existing facilities of the OpenSSH protocol stack, and the proven AsyncSSH python library.

We will look at the code in detail in a few critical places: load a CA safely, authenticate an identity, turn its groups into principals, sign a new key, then inject the result into the agent.

The previous article demonstrates how to operate the thing and what that looks like. This one is a code tour.

CA

ski ca init enters through initialize_ca() in src/ski/ca_commands.py . It delegates file creation to CAFileWriter, then records the public key, fingerprint, and active state in SQLite.

The key generation itself is plain AsyncSSH:

207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
def _generate(self) -> GeneratedCAMaterial:
 try:
 key = self._key_generator()
 if key.get_algorithm() != "ssh-ed25519":
 raise CAFileError("CA algorithm is unsupported")
 private_bytes = key.export_private_key()
 public_bytes = key.export_public_key()
 except CAFileError:
 raise
 except Exception as exc:
 raise CAFileError("CA key generation failed") from exc
 return GeneratedCAMaterial(
 private_key=key,
 private_bytes=private_bytes,
 public_bytes=public_bytes,
 fingerprint=key.get_fingerprint(),
 )

CAFileWriter.install() writes private, public, and KRL files via temporary files and atomic renames. At startup, load_validated_active_ca() in src/ski/ca.py checks ownership and modes, imports both key files, checks that they match, then checks them against the active CA database record. Only then does the runtime receive a ValidatedActiveCA containing an AsyncSSH private-key object capable of signing.

That last point matters: most of the issuer sees a validated object, not a path to a magic key file.

SSH authentication

AsyncSSH provides the SSH server framework. IssuerServer.start() creates an AsyncSSH listener with agent_forwarding=True; _IssuerSSHServer implements the server-side authentication callbacks. It offers exactly one keyboard-interactive exchange with a password and a TOTP prompt.

After receiving the answers, validate_kbdint_response() does the meaningful work:

140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
try:
 canonical_username = self._identity_store.lookup_identity(username)
 password_ok = self._identity_store.verify_password(
 canonical_username,
 responses[0],
 )
 totp_ok = self._identity_store.verify_totp(
 canonical_username,
 responses[1],
 now=int(self._clock()),
 )
 if not password_ok or not totp_ok:
 if self._connection is not None:
 self._connection.abort()
 return False
 self._authenticated_identity = self._identity_store.get_group_snapshot(
 canonical_username,
 )
 return True

The server canonicalizes the username from the SSH banner and input by looking it up in the identity store. It then verifies both the password and TOTP factor. The group set assigned to the user is captured as a group snapshot. This becomes the input to issuance, that is, the creation key and cert.

We do, in theory, support pluggable identification backends, and provide only one demo backend using an SQLite database. How that works is discussed below, the important functions being called are available in identities.py:368, verify_password() and identities.py:402, verify_totp(). We are using pyotp for this job:

402
403
404
405
406
407
408
409
410
411
412
413
 def verify_totp(self, username: str, code: str, *, now: int | None = None) -> bool:
 try:
 record = self.get_user(username)
 if not record.enabled or not isinstance(code, str):
 return False
 totp = pyotp.TOTP(record.totp_secret)
 if self._totp_verifier is not None:
 return bool(self._totp_verifier(totp, code, now=now))
 for_time = None if now is None else datetime.fromtimestamp(now, tz=UTC)
 return bool(totp.verify(code, valid_window=1, for_time=for_time))
 except (IdentityStoreError, ValueError, TypeError):
 return False

In order to succeed, we do need a session with agent-forwarding to be able to inject the generated cert. That is why the session implementation, _IssuerSession._run_request() in src/ski/server.py , also rejects the request if AsyncSSH reports no forwarded agent path. No point in running an issuance with no delivery channel.

Identity abstraction

The SSH server depends on this deliberately small protocol, not on SQLite:

123
124
125
126
127
128
129
class IssuerIdentityProvider(
 IdentityAuthenticator,
 CanonicalIdentityLookup,
 GroupSnapshotProvider,
 Protocol,
):
 """Combined narrow read-only capability required by the SSH issuer."""

The three parent protocols are the contract: canonical lookup, verification of the two existing factors, and a current group snapshot. It is a good boundary for a production adapter backed by LDAP, Active Directory, Keycloak, Okta, or another organisation-specific identity service and contains no write-operations.

The current demonstrator does not use any of that so it can be self-contained. ServiceRuntime.start() constructs SqliteIdentityStore directly. Its get_group_snapshot() reads the local record, rejects disabled users, and returns the username plus groups. The ski user and ski group commands exist to administer that demo backend. They should not be part of a production issuer: identity and group management belongs in the upstream identity system, and the issuer should be read-only with respect to it.

Principal construction

The certificate receives a structured identity record.

build_principals() in src/ski/policy.py accepts only canonical values and produces a compact, predictable principal set:

47
48
49
50
51
52
def build_principals(username: object, groups: Sequence[object]) -> tuple[str, ...]:
 """Build the canonical user and group principals for one identity."""
 canonical_username = validate_username(username)
 canonical_groups = tuple(validate_group_name(group) for group in groups)
 principals = (canonical_username, *(f"group:{group}" for group in canonical_groups))
 return validate_principals(principals)

For kris in prod, this produces ("kris", "group:prod"). The first principal can support self-login policy; subsequent group: principals let a target host make local group-based authorization decisions. The grammar and duplicate checks prevent principals from becoming an accidental free-form authorization language.

Signing

OrdinaryCertificateFactory.issue() in src/ski/credentials.py is the centre of the issuer. It allocates a random 64-bit serial, sets the fixed 25-hour validity window, creates a fresh user key, and invokes the CA private key:

 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
serial = self._serial_allocator()
if not isinstance(serial, int) or not 0 <= serial < 2**64:
 raise StateError("certificate serial is malformed")
valid_after = int(self._clock())
valid_before = valid_after + ORDINARY_CERTIFICATE_LIFETIME
comment = (
 f"ski:{identity.username}:{self._active_ca.record.fingerprint}:{serial}"
)
private_key = asyncssh.generate_private_key(
 "ssh-ed25519",
 comment=comment,
)
flags = {
 flag: extension in self._extensions
 for extension, flag in ORDINARY_EXTENSION_FLAGS.items()
}
certificate = self._active_ca.private_key.generate_user_certificate(
 private_key,
 identity.username,
 serial=serial,
 principals=principals,
 valid_after=valid_after,
 valid_before=valid_before,
 permit_x11_forwarding=flags["permit_x11_forwarding"],
 permit_agent_forwarding=flags["permit_agent_forwarding"],
 permit_port_forwarding=flags["permit_port_forwarding"],
 permit_pty=flags["permit_pty"],
 permit_user_rc=flags["permit_user_rc"],
 touch_required=True,
 comment=comment,
)

Again, we make use of AsyncSSH library functions for certficiate handling: generate_user_certificate() does the OpenSSH certificate signing. Its first argument is the new user key, it contains both the private and public bytes; AsyncSSH uses only the public half. The remaining arguments become the certificate’s claims and restrictions. The configured extensions are converted into explicit OpenSSH permit flags instead of being copied as arbitrary text.

OrdinaryIssuanceService.commit() persists only safe issuance evidence: serial, identity, public-key fingerprint, principals, validity period, and request ID.

It does not persist the ephemeral private key.

Injection

Signing has produced Python objects; it has not yet delivered anything to the user. OrdinaryAgentInjector.handle() connects to the agent forwarded by the current AsyncSSH server connection, removes only an earlier credential it can recognize as its own, and then loads the new pair:

190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
async with self._agent_factory(connection, self._active_ca) as agent:
 owned = await agent.owned_keys(identity)
 removed_owned = False
 credential: OrdinaryIdentity | None = None
 try:
 for _ in range(5):
 credential = self._issuance.prepare(identity)
 lifetime = max(
 1,
 credential.valid_before - int(self._clock()),
 )
 if owned and not removed_owned:
 await agent.remove_keys(owned)
 removed_owned = True
 await agent.add_credential(credential, lifetime=lifetime)
 try:
 record = self._issuance.commit(
 credential,
 request_id=request_id,
 )
 except DuplicateCertificateSerialError:
 cleanup = await agent.remove_credential(credential)
 if not cleanup.complete:
 raise StateError("credential cleanup failed")
 continue
 return OrdinaryInjectionResult(
 credential=credential,
 record=record,
 groups=identity.groups,
 )

The adapter behind agent uses asyncssh.connect_agent(connection). AsyncSSH opens the forwarded-agent connection and speaks the OpenSSH agent protocol; the application code works with keys and certificates instead of framing agent-protocol messages itself.

The ordering is worth noticing. The new credential is first added with a lifetime no later than certificate expiry, then durable issuance evidence is committed. A duplicate serial triggers targeted agent cleanup and a retry; other failures also attempt cleanup. That is the small but important piece which prevents an in-memory credential from silently surviving a failed issuance transaction.

The whole issuer is therefore ordinary Python around a few AsyncSSH boundary calls: generate a private key, sign a user certificate, operate the forwarded agent. The surrounding code establishes what those calls are allowed to mean.

Top

Installing the ephemeral SSH key demonstrator

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

The previous article explained ephemeral SSH keys. This is the concrete demonstrator: a Mac runs the certificate issuer, a Rocky Linux VM is the production host, and a normal ssh login uses a one-day certificate held only in the Mac’s ssh-agent.

Mac: issuer Rocky Linux: target host
──────────────────────────────────────────── ─────────────────────────────────────
ski serve sshd
 ├── user CA private key ├── public user-CA key
 ├── local identity and group data ├── local authorization.toml
 └── issues a short-lived certificate └── ski-authorize helper
 │ ▲
 │ agent forwarding during issuance │ certificate login
 ▼ │
 ssh-agent on the Mac ─────────────────────────────────────────┘
 ephemeral private key + certificate

The CA private key, issuer database, and user private key remain on the Mac. The Rocky VM receives only the CA public key and its own local authorization policy. Its login path does not contact the issuer.

This is a demonstrator, not a production deployment guide. In a real setup, verify host keys through a trusted channel, protect the issuer’s CA key, and deploy target-host files with configuration management.

More importantly, the issuer’s SQLite database is demonstrator scaffolding, not a production identity source. A real issuer would read canonical users and group memberships from the organisation’s identity backend: for example LDAP, Active Directory, Keycloak, or Okta. User creation, passwords, second factors, and group administration remain the responsibility of that backend. The issuer is read-only with respect to identity data: after authenticating a request and reading a current group snapshot, it issues a certificate. It does not manage users or mutate group membership through ski commands.

1. Prepare the issuer on macOS

The checkout on the Mac is ~/PycharmProjects/ski and its dependencies are locked through uv:

~ $ cd PycharmProjects
PycharmProjects $ git clone https://github.com/isotopp/server-key-injection/ ski
PycharmProjects $ cd ski/
ski $ ls
AGENTS.md README.md
developer docs
packages pyproject.toml
src tests
uv.lock

The local .env selects paths. It contains no secret material: the private CA key is a file at the named path, not a value in the environment file.

SKI_CA_DATABASE=ski.sqlite3
SKI_CA_PRIVATE_KEY=etc/keys/user_ca
SKI_CA_PUBLIC_KEY=etc/keys/user_ca.pub
SKI_CA_KRL=etc/keys/revoked.krl
ORDINARY_CERT_EXTENSIONS=pty

ORDINARY_CERT_EXTENSIONS=pty deliberately grants a terminal but not agent, TCP, or X11 forwarding to the resulting production login. Agent forwarding is needed only for the brief connection to the issuer, to put the new credential in the workstation’s agent.

2. Create the user CA

Initialize a User CA once. This generates an Ed25519 CA keypair and records its state in the configured database:

ski $ mkdir -p etc/keys
ski $ uv run ski ca init
CA initialized.
Algorithm: ssh-ed25519
Fingerprint: SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q
CA committed, but service notification failed; retry notification.
ski: CA initialized; service notification failed; retry notification

ski $ uv run ski ca show
CA ID: 1
Algorithm: ssh-ed25519
Fingerprint: SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q
Status: active
Activated: 1787422938

ski $ uv run ski ca public-key
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILGt49Riwdjygm6Okt1Uvd9ldgNaFRrpb8OZhyCktJQR

There is no running issuer yet, so the service-notification warning is expected. Keep both values. The target policy pins the CA by fingerprint; user-ca.pub is the public verification key given to sshd. Neither is secret. The private CA key in etc/keys/user_ca must never leave the issuer.

3. Create a user and group

Create kris and make it a member of prod:

ski $ uv run ski user add kris
Password:
User created: kris
TOTP secret: <redacted>
TOTP URI: otpauth://totp/ski:kris?secret=<redacted>&issuer=ski
ski: service notification failed; mutation committed; retry notification

ski $ uv run ski group add prod
Group created: prod
ski: service notification failed; mutation committed; retry notification

ski $ uv run ski group member add prod kris
Membership added: prod kris
ski: service notification failed; mutation committed; retry notification

Those notifications also have nowhere to go until ski serve starts. Password and TOTP authenticate the issuance request. The outcome is a certificate with the signed principal group:prod; the password is not used on the target host.

For this demonstrator, we added the TOTP secret into a TOTP generator such as bitwarden. Another way to work with this is to use oath-toolkit:

$ brew install oath-toolkit
$ oathtool -b <redacted>
459274

4. Start the issuer and obtain a certificate

Run the issuer in one terminal:

ski $ uv run ski serve --port 2222
service_starting: ski startup requested
service_ready: ski is ready

In another terminal, ensure a local ssh-agent exists, then forward it only to this trusted local issuer:

ski $ ssh -A -tt -p 2222 \
 -o StrictHostKeyChecking=no \
 -o UserKnownHostsFile=/dev/null \
 kris@127.0.0.1
Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts.
ski
Authenticate with your ski password and TOTP code.
(kris@127.0.0.1) Password:
(kris@127.0.0.1) 2FA:
Key loaded: kris serial=367065752294255925 valid-until=1787513156
Groups: prod
Connection to 127.0.0.1 closed.

The host-key options are acceptable only for this disposable loopback test: they disable SSH’s protection against connecting to the wrong server. A real issuer needs a verified known_hosts entry.

The issuer authenticates the user, generates a fresh keypair, signs the public key, and sends an add-identity request through the forwarded agent channel. It then closes the interactive connection. The local agent now shows one key and its certificate:

ski $ ssh-add -l
...
256 SHA256:uQLZAupxPTPBXhEMMWW6UzsH3JfFNIVS1Fl9xkOgg+U ski:kris:SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q:367065752294255925 (ED25519-CERT)
256 SHA256:uQLZAupxPTPBXhEMMWW6UzsH3JfFNIVS1Fl9xkOgg+U ski:kris:SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q:367065752294255925 (ED25519)

These are one credential, not two: the ED25519 entry is the ephemeral private key, while ED25519-CERT is its signed public certificate. They have the same fingerprint. Nothing new was written to ~/.ssh.

At this point the issuer and agent work. No target host has yet been taught to trust the certificate.

5. Install the target-host authorizer

The target is a Rocky Linux VM with a pre-existing local Unix account named kris. The authorizer does not create accounts or map the certificate to a different account. Check for OpenSSH 9 or later, then run the packaged installer as root from packages/ski-authorize:

[root@localhost ski-authorize]# git clone https://github.com/isotopp/server-key-injection/
...

[root@localhost ski-authorize]# cd server-key-injection
[root@localhost server-key-injection]# cd packages/ski-authorize
[root@localhost ski-authorize]# ./install.sh
Python 3.12 is already installed
Resolved 6 packages in 121ms
Building ski-authorize @ file:///root/server-key-injection/packages/ski-authorize
Prepared 1 package in 14ms
Installed 1 executable: ski-authorize
warning: `/opt/ski-authorize/bin` is not on your PATH.
ski-authorize installed below /opt/ski-authorize

The PATH warning is irrelevant: sshd invokes the helper by absolute path. The installation tree is root owned; the helper runs under an unprivileged ski-authz account (which was already created by the installer):

/opt/ski-authorize/
├── bin/ski-authorize
├── cache/
├── config/
│ ├── authorization.toml
│ └── user-ca.pub
├── python/
└── tools/

Transfer only the public CA key from the Mac through a reviewed mechanism. Never transfer the private CA key, SQLite database, .env, or agent contents.

Put the output of uv run ski ca show and uv run ski ca public-key into the host’s local policy and verification-key file:

[ssh]
trusted_ca_fingerprint = "SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q"
allowed_groups = [ "group:prod" ]
allow_self_login_only = true
[root@localhost config]# pwd
/opt/ski-authorize/config
[root@localhost config]# cat user-ca.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILGt49Riwdjygm6Okt1Uvd9ldgNaFRrpb8OZhyCktJQR
[root@localhost config]# ls -l
total 8
-rw-r-----. 1 root ski-authz 375 Aug 22 20:30 authorization.toml
-rw-r--r--. 1 root root 81 Aug 22 20:32 user-ca.pub

Compare the fingerprint using an independent channel before accepting it.

allow_self_login_only = true permits a certificate for kris only to log into local account kris (other configurations are not supported anyway at this point int time).

The group requirement adds group:prod to the host: Only users with group:prod in their certficate may log in.

6. Configure and reload sshd

The installer places this fragment in /etc/ssh/sshd_config.d:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
TrustedUserCAKeys /opt/ski-authorize/config/user-ca.pub
CASignatureAlgorithms ssh-ed25519
AuthorizedPrincipalsCommand /opt/ski-authorize/bin/ski-authorize --config /opt/ski-authorize/config/authorization.toml --ca-fingerprint %F %u %t %k
AuthorizedPrincipalsCommandUser ski-authz

AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no

TrustedUserCAKeys names the CA allowed to sign user certificates. AuthorizedPrincipalsCommand receives the signing CA fingerprint (%F), requested local user (%u), certificate type (%t), and certificate body (%k). sshd verifies the cryptography; the helper applies the local policy. The forwarding directives constrain the production session, not the earlier connection to the issuer.

Validate the whole SSH configuration before reloading:

[root@localhost ~]# sshd -t
[root@localhost ~]# systemctl reload sshd
[root@localhost ~]# logout
flowchart TB
CERT["Certificate offered by client\nkey ID: kris\nprincipal: group:prod"] --> SSHD["Rocky sshd"]
CA["user-ca.pub"] --> SSHD
SSHD -->|"signature valid"| AUTHZ["ski-authorize\nrun as ski-authz"]
POLICY["authorization.toml\nCA fingerprint, group:prod,\nself-login only"] --> AUTHZ
AUTHZ -->|"principal kris allowed"| SSHD
SSHD --> SESSION["Unix session for kris"]

7. Login and inspect the evidence

The Mac SSH client discovers the certificate-backed key in its agent. No special IdentityFile is needed:

~ $ ssh 192.168.64.3 -l kris
Last login: Sat Aug 22 20:28:45 2026 from 192.168.64.1
[kris@localhost ~]$

The host log proves that this was certificate authentication:

[root@localhost log]# tail -4 /var/log/secure
Aug 22 20:35:20 localhost sshd-session[2152]: Accepted publickey for kris from 192.168.64.1 port 60378 ssh2: ED25519-CERT SHA256:yutOUApby5l70x4kxF1QYwXCnrygAWEVsZYiEmuoz+s ID kris (serial 9739375750096054870) CA ED25519 SHA256:r2+zadY0Ln3LQiNrCFxNY5aVVqSFuuDtCDEwc5itj+Q
Aug 22 20:35:20 localhost sshd-session[2152]: pam_unix(sshd:session): session opened for user kris(uid=1000) by kris(uid=0)

The useful evidence is ED25519-CERT, the key ID, serial, and CA fingerprint. The target verified a certificate signed by the expected CA and opened its local kris account. The issuer did not take part in this login.

8. Treat the SELinux AVC as a finding

The successful Rocky login also produced an AVC for ski-authorize trying to execute ldconfig from sshd_session_t. The helper still completed because native-library discovery has a fallback. That is not a reason to suppress the denial. Do not disable SELinux, run audit2allow. Fix packaging, runtime dependency discovery, and file labels so the helper works under its intended confinement.

The end state is deliberately small: the Mac agent contains a short-lived private key and certificate; the Rocky host accepts it only when its local CA and group policy agree. Once the agent entry or certificate expires, new logins stop until the user obtains a fresh credential from the issuer.

Top

Ephemeral SSH Keys

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

The SSH keys living in your $HOME/.ssh are practically immortal. You create id_ed25519 once, copy its public half to a target host and insert it into authorized_keys, and retain access for years. The private key lives on a laptop, in backups, and often on machines which have long since stopped being used.

That is convenient, but it makes for an unpleasant authorization model. When someone leaves the company or a team, every relevant host must be changed. If a laptop is lost, someone must know where its key is authorized. And authorized_keys becomes a distributed, hard-to-audit identity database.

A better separation is to stop having hosts trust every individual user key. Instead, they trust a small number of SSH certificate authorities (CAs). A CA issues an SSH certificate for a fresh, short-lived key. The certificate might be valid for only 25 hours. Its private key lives solely in ssh-agent and then disappears. It is never materialized on a developer’s laptop disk.

This is not a new SSH protocol. OpenSSH has supported it for a long time (since OpenSSH 8.8). The important part is understanding what the key, certificate, agent, and sshd each do.

Three things which are not the same

Classic public-key authentication has one key pair:

  • The private key stays with the user and produces signatures.
  • The public key lives in authorized_keys on the target host.

SSH certificates add a third object: the user certificate. It contains a public user key, a list of principals, a validity period, a serial number, and optional restrictions. An SSH CA signs this package with its private CA key.

The certificate is not secret. Nor is it a replacement for the private key. It merely says: “The holder of this public key may act as these principals between these times.” During login, the client must still prove that it holds the matching private key.

flowchart TB
CA["SSH user CA\nprivate CA key"] -->|"signs"| CERT["SSH user certificate\npublic key\nprincipals, expiry"]
KEY["ephemeral private\nuser key"] -->|"matches"| CERT
KEY -->|"proof-of-possession signature"| SSHD["sshd on the target host"]
CERT -->|"CA signature and policy"| SSHD
CAPUB["public CA key\non target host"] -->|"trust anchor"| SSHD

The central private CA key is especially sensitive: anyone who controls it can issue certificates. The temporary private user key is shorter lived, but it is still a real login secret for as long as it remains valid. It belongs neither in a Git repository nor a shell history nor permanently in ~/.ssh – and it usually isn’t, because we’ll put it into a developers agent, which will not let to go to a disk.

What ssh-agent actually does

ssh-agent is a process on the workstation. It keeps private keys in memory and exposes a Unix socket. The path to this socket is in SSH_AUTH_SOCK. Programs such as ssh, scp, and git talk to the agent instead of reading the key file themselves.

In this setup, the SSH client is never handed the private key. Instead it asks the agent: “Here are the authentication-protocol data; can you sign them with key X?” The agent returns the signature. This permits local passphrase prompts, hardware-backed keys, or a fixed lifetime without every SSH client needing to understand those details.

sequenceDiagram
participant C as ssh client
participant A as ssh-agent
participant S as target host:sshd
C->>A: Which public identities do you have?
A-->>C: Public keys and certificates
C->>S: Offer certificate and public key
S-->>C: This identity is allowed. Sign the session
C->>A: Sign the SSH session data with the matching key
A-->>C: Signature
C->>S: Signature
S-->>C: Login accepted

Normally, one loads a long-lived key into the agent with ssh-add.

In the model with ephemeral keys, a new key is generated by the remote issuer and tunneled into the agent through the local ssh client, together with its matching certificate. This ephemeral key and cert are given a lifetime.

ssh-add -l will then show both: an ED25519 entry and an ED25519-CERT entry.

$ ssh-add -l
...
256 SHA256:RquDquM94ouXwDMCo4uNdWiuGz1nwEzvX7Qmwev80QI ski:kris:SHA256:w6aQ5C02+aexz6g5eULg7TSO1piXyAD90+fy259e5fg:13861769879096579493 (ED25519-CERT)
256 SHA256:RquDquM94ouXwDMCo4uNdWiuGz1nwEzvX7Qmwev80QI ski:kris:SHA256:w6aQ5C02+aexz6g5eULg7TSO1piXyAD90+fy259e5fg:13861769879096579493 (ED25519)

These are not two independent credentials: the first is the private key, the second is its signed public certificate.

When the lifetime expires, the agent removes the identity. The private key is then no longer present. It can also be removed explicitly; ssh-add -D is often too broad because it removes every identity from the agent.

How the key reaches the agent

In our setup, we have the “issuer”, a server into which the user logs at the beginning of the work day.

The issuer verifies the user password and a TOTP 2FA challenge, then creates a fresh Ed25519 key pair with a 25h lifetime, the users user-principal and the set of groups the user is in, plus the capabilities of the key (agent forwarding, port forwarding and so on). It signs the public half with the user CA, and puts both into the workstation’s agent. It then closes the connection.

For the issuer to reach the local agent, the agent is deliberately forwarded to this one host:

ssh -A kris@ssh.example.com

This is what happens at the protocol level:

  1. The client sends an auth-agent-req@openssh.com request on the SSH session channel. This asks the remote sshd to make the forwarded agent available to this login session.
  2. sshd creates a Unix-domain listening socket owned by the logged-in user, sets SSH_AUTH_SOCK in that user’s session environment, and gives the process restrictive access to the socket.
  3. When a process on the issuer connects to that socket, sshd opens an auth-agent@openssh.com channel back to the client. It does not send a private key and it does not create a second agent on the server.
  4. The client connects to its local SSH_AUTH_SOCK and copies the byte stream between that socket and the auth-agent@openssh.com channel.

The result is a short-lived proxy. To an issuer process, the socket named by SSH_AUTH_SOCK looks like an ordinary local OpenSSH agent socket. The real agent, and the private keys already stored in it, stay on the workstation.

An agent socket transports the SSH agent protocol, not SSH packets. Individual messages are length-prefixed and start with a request or reply type. In this case, the issuer uses an add-identity request — normally the constrained form, SSH_AGENTC_ADD_ID_CONSTRAINED — containing the newly generated private key, the certificate associated with its public half, and a lifetime constraint. The exact message bytes cross the remote socket, the SSH channel, and the local socket before reaching the local agent unchanged. The reply follows the same route in reverse.

sequenceDiagram
participant A as workstation:ssh-agent
participant C as workstation:ssh -A
participant D as issuer:sshd
participant I as issuer process
C->>D: Session channel request: auth-agent-req@openssh.com
D-->>C: Request succeeds
Note over D: Create Unix socket and set SSH_AUTH_SOCK
D->>I: Start login session
I->>D: Connect to remote SSH_AUTH_SOCK
D->>C: Open auth-agent@openssh.com channel
C->>A: Connect to local SSH_AUTH_SOCK
I->>D: SSH_AGENTC_ADD_ID_CONSTRAINED message
D->>C: Agent-protocol bytes
C->>A: Same agent-protocol bytes
A-->>C: SSH_AGENT_SUCCESS
C-->>D: Agent-protocol reply
D-->>I: Reply from remote socket

The channel is opened for each connection to the remote socket. It is a stream proxy, so an issuer can first request identities, then request signatures, or add an identity. The forwarded connection ends when the socket connection or SSH session ends; the credential loaded into the local agent survives only for the lifetime set in the add-identity request.

This pattern has an important trust assumption: agent forwarding does not belong on arbitrary servers. Whoever may use a forwarded agent can request login signatures from its keys. In this case, the issuer may also add a new identity to the agent. That is the intended effect, but it is justifiable only for a particularly trusted, reviewed issuer.

The alternative — having the issuer return a private key as a file — is worse: it produces files, backups, and copies. Agent injection is no magic, but it limits the key to memory and a defined lifetime.

What the production host verifies

On a conventional host, an entry in ~/.ssh/authorized_keys means that one public key may log in as one local Unix account. Certificates move the trust anchor to the CA key:

TrustedUserCAKeys /etc/ssh/user-ca.pub

sshd can now verify an offered user certificate by itself:

  1. Is it really a user certificate, and does its public key match the key now proving possession?
  2. Is the signature valid under a configured CA key?
  3. Does the local clock fall between valid_after and valid_before?
  4. Does one of the certificate principals match the requested local login name or the local principal policy?
  5. Do critical options or a revocation list prevent login?

Cryptography therefore does not answer the whole question, “May Kris use this host?” It answers only: “A trusted CA issued this certificate for this key and these claims.” The host policy then decides whether any of those claims is sufficient here.

flowchart TD
OFFER["Client offers\nprivate key + certificate"] --> SIG{"Proof of possession\nvalid?"}
SIG -->|no| DENY["Deny login"]
SIG -->|yes| CA{"CA signature\ntrusted?"}
CA -->|no| DENY
CA -->|yes| TIME{"Certificate\nstill valid?"}
TIME -->|no| DENY
TIME -->|yes| POLICY{"Principal and local\nhost policy match?"}
POLICY -->|no| DENY
POLICY -->|yes| OK["Open session for\nlocal account"]

The local policy can be simple: the kris principal may use the kris Unix account. It may also use groups as principals, such as group:prod. An AuthorizedPrincipalsCommand can then compare a certificate’s signed group against a local, root-protected policy to decide whether that group grants access on this host. The host need not query the issuer, LDAP, Okta, or any other identity service.

The AuthorizedPrincipalsCommand does not need to run as root or the ssh user, and in fact it should not. It also does not need to own any files at all, so it can’t overwrite anything. All communication is through pipelines and command line parameters.

That is operationally useful. The production host needs only the public CA key and its local policy. There may even deliberately be no route from the production network to the issuer. The user carries their fresh, signed certificate across the firewall.

Short lifetime is the revocation strategy

A certificate with a 25-hour lifetime has a clear disadvantage: if someone is removed from a group at 09:00, a certificate issued at 08:59 may still open new sessions until it expires. The agent cannot “recall” a certificate merely because central group membership has changed.

This is not a bug; it follows from hosts operating offline. The maximum remaining lifetime is the deliberately chosen risk window. Shorter certificates reduce it, but require renewal more often. Additional Key Revocation Lists (KRLs) can invalidate individual certificates early, but must then be reliably distributed to every host too.

Time synchronization and an explicit lifetime policy are therefore security features. A 25-hour period can cover a working day and the next morning’s renewal window; other environments may use four hours, one shift, or a few minutes. “Short” is not a cryptographic property. It is a trade-off between operations and the exposure window.

What this improves — and what it does not

Ephemeral SSH keys reduce the lifetime of a compromisable user secret and replace a large collection of distributed authorized_keys entries with a few CA trust anchors. Group and authorization changes take effect at the next issuance, without every host having to make a central query at login.

They do not replace normal SSH fundamentals:

  • The workstation must still verify the issuer’s host key. A user CA is not the issuer’s host key.
  • The private CA key needs much stronger protection than an ordinary user key.
  • Agent forwarding is a powerful capability and must be targeted only at trusted issuers.
  • Target hosts must correctly manage their public CA key, local policy, and clock.

The model does not remove trust; it makes it visible: in a small CA, a tightly bounded issuer, the local agent, and an explicit policy on every target host. The user still works with ordinary ssh. Only the key used to identify them has a short, intentionally ephemeral life.

This description is based on the architecture and operational documentation of server-key-injection .

Top

FreeBSD 14.5-BETA3 Available

Post by FreeBSD Newsflash via FreeBSD News Flash »

The third BETA build for the FreeBSD 14.5 release cycle is now available. ISO images for the amd64, i386, armv7, aarch64, powerpc, powerpcspe, powerpc64, powerpc64le, and riscv64 architectures are FreeBSD mirror sites.
Top

FreeBSD Foundation Intern Jim Huang Chen on Raspberry Pi Support and Open Source

Post by FreeBSD Foundation via FreeBSD Foundation »

When Jim Huang Chen began his internship with the FreeBSD Foundation, he was only finishing his first year as a Software Engineering student at the University of Waterloo. A few months later, he had spent the summer working inside the FreeBSD codebase, contributing to Raspberry Pi support, learning from longtime developers, and gaining a much deeper understanding of how an operating system works.

For Jim, that process of learning became one of the most valuable parts of the experience.

From a Graphing Calculator to Systems Programming

Jim’s interest in programming started well before university.

Around seventh grade, he began programming simple games on a TI-84 graphing calculator using TI-BASIC. That eventually led him to the Cemetech community, where he connected with other technology enthusiasts and started exploring lower-level languages, including assembly and C.

Throughout high school, Jim frequently worked with older hardware that he didn’t want to purchase Windows licenses for. Linux became the natural alternative, introducing him more deeply to open source software and the communities surrounding it.

“Systems programming is about as close as you can get to treating computers as a machine rather than magical software abstractions,” Jim said.

That perspective appealed to his interests in mathematics, computation, and understanding what computers are actually doing beneath the software we interact with every day.

Taking a Chance on FreeBSD

Jim originally expected to pursue Google Summer of Code and didn’t know that the FreeBSD Foundation offered an internship program.

That changed when he came across the opportunity on WaterlooWorks, the University of Waterloo’s job board.

With hundreds of other applicants, he wasn’t particularly confident about his chances. But the work interested him enough to apply anyway.

“I figured I might as well write a cover letter, submit my resume, and pray,” he said. “I’m glad it worked out!”

That application led to a summer focused primarily on two Raspberry Pi projects.

Expanding FreeBSD Support for Raspberry Pi

Jim’s first project involved porting Raspberry Pi Imager to FreeBSD.

Raspberry Pi Imager provides a graphical way for users to prepare a Raspberry Pi, including selecting a device, choosing an operating system image, configuring settings, and writing the image to storage.

Before Jim’s work, the application supported Linux, macOS, and Windows, but not FreeBSD natively.

His task was to add a FreeBSD backend using FreeBSD-specific libraries and functionality. The goal was to make a native version available through the FreeBSD Ports Collection rather than relying on another operating system’s version through a compatibility layer. The resulting changes are currently under review for inclusion upstream.

His second project took him even deeper into hardware: bringing FreeBSD support to the Raspberry Pi Compute Module 5.

That work required adding support for components including the Raspberry Pi 5’s PCIe controller and RP1 southbridge, which manages access to peripherals such as Ethernet and USB.

By the time Jim reflected on the project, he was confident that the PCIe controller was working, while memory allocation for devices connected through the RP1 remained an ongoing challenge.

Learning to Understand the “Why”

For Jim, the biggest reward wasn’t one patch or technical milestone.

It was being able to look back and see how much more he understood.

As a first-year student entering a large operating system codebase, there were inevitably pieces that initially didn’t make sense. Over the course of the internship, reading code, writing code, studying documentation, and asking questions gradually changed that.

He reached a point where he could not only explain what his code was doing, but why it needed to be there.

That growth also changed some of his assumptions about FreeBSD.

Before the internship, Jim viewed FreeBSD largely as an operating system for a relatively stable world of amd64 servers. Working with arm64 showed him how much development is happening underneath FreeBSD’s consistent user-facing experience.

He encountered evolving interrupt and clock frameworks, different approaches to device enumeration, and interfaces that have changed considerably over the Project’s history.

“Just taking time to explore how much is boiling in the code under the always-calm, always-consistent user-accessible surface of the operating system has been pretty amazing,” he said.

Finding the Problem Before Finding the Solution

One of the most important lessons Jim learned was that solving a technical problem often starts with figuring out exactly what the problem is.

For the Raspberry Pi Imager work, FreeBSD’s documentation gave him a strong starting point. Man pages, the FreeBSD Handbook, existing scripts, and other resources helped him gradually understand FreeBSD’s disk architecture.

The Raspberry Pi Compute Module 5 was more difficult.

Because components such as Broadcom’s BCM2712 and Raspberry Pi’s RP1 are relatively new, detailed technical documentation could be difficult to find. Jim supplemented what was available by reading about driver development, studying similar drivers, and asking experienced developers for guidance.

“More often than not, it’s easier to ask someone why something needs to be done than to wizard out the intention from the code they write,” he said.

He also learned to make his development process more efficient.

Early in the internship, rebuilding the FreeBSD kernel locally could take minutes for an incremental build and hours for a clean build. Moving compilation to the Foundation’s build servers reduced those waits significantly.

Jim then automated repetitive steps in his workflow, including building, booting the Raspberry Pi, mounting disks, copying files, and unmounting them.

Those improvements did more than save time. They made experimentation easier and gave him more room to focus on understanding the underlying technical problems.

 

Meeting the Community Behind the Code

Jim also attended BSDCan and the FreeBSD Developer Summit during his internship.

Initially, the experience was intimidating.

Only about a month into the internship, Jim found himself surrounded by people who had been developing and using FreeBSD for decades. But that feeling changed as he began talking with community members.

“If you have something to talk about, it’s more likely than not that someone will be excited to talk you through it and discuss whatever you had on your mind,” he said.

He attended sessions covering topics ranging from security and capability pointers to static analysis, formal verification, hardware support, and the future of FreeBSD’s scheduler.

What surprised him most was how social the events were. Rather than simply listening to presentations, much of the value came from discussions, questions, feedback, and conversations with other attendees.

“Definitely much more immersive than the 9-5 university-style lectures I anticipated.”

Making Open Source Feel More Accessible

Working directly with the FreeBSD community also changed Jim’s perception of contributing to open source.

From the outside, large projects can sometimes appear intimidating or reserved for highly experienced developers. His experience with FreeBSD was different.

“The people are nice,” he said. “They are willing to give you mentorship and to give some thought to patches you submit.”

The internship also gave him a greater appreciation for the broader open source ecosystem and how much software depends on projects maintained by relatively small groups of contributors.

For Jim, that highlighted both the importance of open source and the need for more people to contribute their time, expertise, and support.

His Advice: Start the Conversation

Jim’s advice for students thinking about contributing to FreeBSD is straightforward: reach out.

“I think that if you’re thinking about getting involved yourself, you should really bite the bullet and email someone involved in the project.”

Using FreeBSD itself can also reveal opportunities to contribute. If something doesn’t work as expected, investigating why could lead to a first bug fix or patch.

For those interested in a particular area, Jim recommends looking through development activity to find the people working on that part of the system and asking where help is needed.

The important part is starting somewhere.

Plenty More to Explore

Jim doesn’t plan for his FreeBSD work to end with the internship.

He hopes to continue improving Raspberry Pi 5 and RP1 support, create ports for software he would like to use on FreeBSD, and investigate issues he has encountered while running FreeBSD on his own hardware.

He is also interested in eventually exploring the FreeBSD network stack, scheduling, and resource management.

As for his career, Jim is still early in his studies and keeping his options open, but systems programming and formal methods are areas he would like to continue exploring.

A Summer of Exploration

Jim credits Ed Maste and Siva Mahadevan for giving him the opportunity and supporting him throughout the internship. He also thanked Bjoern Zeeb, Adrian Chadd, Sheng-Yi Hung, and Minsoo Choo, along with fellow interns Sourojeet Adhikari and Nimish Jain, for their guidance, technical help, and willingness to exchange ideas.

The FreeBSD Foundation would also like to thank Jim for his contributions throughout the semester. We appreciate the curiosity, dedication, and persistence he brought to his work, as well as his willingness to take on new challenges and explore different areas of the FreeBSD Project.

Looking back, Jim describes the internship as challenging at times, but ultimately an opportunity to learn in a way that would be difficult to replicate elsewhere.

He appreciated the flexibility of the work, the emphasis on independent exploration, and the freedom to experiment without being afraid of getting everything right immediately.

“I feel that the time I got to pursue the bits I’m interested in, to try things out without fear of repercussions, and to pore over technical goodies was a valuable experience for my first internship,” he said.

For someone who started programming by experimenting with a graphing calculator, that curiosity remains at the center of Jim’s work.

Only now, the system he’s exploring is a little bigger.

The post FreeBSD Foundation Intern Jim Huang Chen on Raspberry Pi Support and Open Source first appeared on FreeBSD Foundation.

Top

Service Discovery mit Zookeeper

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

Ein anpaßbares Pattern für Service Discovery, hier am Beispiel MySQL – aber es kann im Grunde für jede andere Flotte von Servern verwendet werden. Ein Dienst veröffentlicht seine Bereitschaft in Zookeeper. Ein lokaler Agent beobachtet diesen Zustand und materialisiert daraus eine Datei. Anwendungen lesen die Datei und müssen Zookeeper weder kennen noch erreichen können; sie arbeiten mit der Datei und müssen diese bei Änderungen neu lesen.

Das Pattern stammt aus dem Betrieb großer MySQL-Installationen bei Booking.com. Es funktioniert genauso für andere Datenbanken, Caches, Message-Broker oder beliebige Pools gleichartiger Dienste.

Der wichtige Punkt ist die Trennung der Aufgaben:

  • Zookeeper ist die konsistente Control Plane für die Mitgliedschaft im Pool.
  • Eine lokale Datei ist die möglicherweise kurz veraltete, aber hoch verfügbare Sicht der Data Plane.
  • Der Übergang zwischen beiden ist ein Reconciliation Loop mit Eventual Convergence.

Woher das Problem kam

Booking.com betrieb MySQL in Replikationsbäumen über drei Rechenzentren. An der Wurzel stand ein Primary. In jedem Rechenzentrum übernahm ein Intermediate Primary die Verteilung der Writes an lokale Leaf Replicas.

flowchart TD
P["Primary in RZ A"]
IA["Intermediate Primary in RZ A"]
IB["Intermediate Primary in RZ B"]
IC["Intermediate Primary in RZ C"]
A1["viele Produktions-Replikas in RZ A"]
B1["viele Produktions-Replikas in RZ B"]
C1["viele Produktions-Replikas in RZ C"]
P --> IA
P -->|"ein Write-Stream über die Langstrecke"| IB
P -->|"ein Write-Stream über die Langstrecke"| IC
IA --> A1
IB --> B1
IC --> C1

Auf diese Weise gingen die Writes nur einmal pro entferntem Rechenzentrum über eine Long-Distance-Verbindung. Der dortige Intermediate Primary verteilte sie lokal weiter. Die Zahl der Leaf Replicas vergrößerte daher nicht die Zahl der WAN-Verbindungen.

Anwendungen schrieben auf den Primary und lasen von lokalen Leaf Replicas. Nicht jede Replica war dafür jederzeit geeignet. Eine Produktions-Replika mußte unter anderem erreichbar sein, laufende Replikation und akzeptablen Replikationsverzug haben und durfte sich nicht in Wartung befinden.

Die Menge geeigneter Server war dynamisch. Neue Maschinen kamen hinzu, alte wurden entfernt, Replikas fielen zurück oder wurden aus einem Pool genommen. Die Clients brauchten also nicht die Liste aller MySQL-Server, sondern die Liste der jetzt für ihren Zweck geeigneten lokalen Endpoints.

Eine ausführlichere Beschreibung der MySQL-Topologie steht in That’s a lot of databases .

Drei Generationen Discovery

Die Discovery durchlief drei Generationen: einen Layer-2-Load-Balancer, DNS und schließlich Zookeeper. Diese Alternativen waren bereits Teil einer System-Design-Frage zu MySQL bei Booking.com . Hier geht es um die dort nur angerissene Zookeeper-Variante.

Layer 2 paßte nicht mehr zum Netz

Der erste Load-Balancer arbeitete auf Layer 2. Das war brauchbar, solange das Netz die dafür benötigten gemeinsamen Broadcast-Domains bereitstellte.

Mit dem Wechsel auf ein geroutetes Leaf-and-Spine-Fabric war diese Annahme nicht mehr gegeben. Virtuelle Adressen mit ARP, VRRP oder ähnlichen Verfahren lassen sich nicht einfach über die Grenzen der L2-Domains verschieben. Man kann um die neue Netzstruktur herum wieder L2-Inseln und zusätzliche Komponenten bauen, aber dann arbeitet die Service Discovery gegen die Netzarchitektur statt mit ihr.

DNS konvergierte schlecht kontrollierbar

DNS beseitigte die Abhängigkeit von einer gemeinsamen L2-Domain, brachte aber eine andere Klasse von Problemen:

  • Autoritative Updates und die Sicht der Clients sind zwei verschiedene Vorgänge.
  • Resolver und Anwendungen cachen mit unterschiedlichen Regeln.
  • Manche Clients lösen einen Namen nur beim Start oder beim Aufbau eines Connection Pools auf.
  • Positive und negative Caches reagieren unterschiedlich auf Änderungen.
  • Das schnelle und verläßliche Entfernen eines ungeeigneten Servers ist schwer.

Auch DNS konvergiert schließlich. Zeitpunkt und Zwischenzustände lassen sich bei schnell veränderlichen Server-Pools jedoch schlecht kontrollieren.

Daher ist man zu einem Zeitpunkt auf Zookeeper umgestiegen, um Server zu registrieren und um Clients zu informieren, wo welche Server erreichbar sind.

Zookeeper-Grundlagen

Zookeeper speichert Daten in einem hierarchischen Baum. Ein Knoten in diesem Baum heißt Znode. Anders als bei einem Unix-Dateisystem kann ein Znode gleichzeitig Daten enthalten und Kinder haben. Der Pfad /services/search kann also Konfiguration enthalten und zugleich Elternknoten für einzelne Search-Server sein.

Znodes können persistent oder ephemeral sein:

  • Ein persistenter Znode bleibt bestehen, bis ihn jemand explizit löscht.
  • Ein ephemerer Znode gehört einer Zookeeper-Session. Er verschwindet, wenn diese Session endet oder abläuft.

Eine Session ist nicht dasselbe wie eine einzelne TCP-Verbindung. Ein Client kann die Verbindung zu einem Mitglied des Zookeeper-Ensembles verlieren und sich mit einem anderen verbinden, ohne seine Session zu verlieren. Während einer kurzen Unterbrechung bleiben seine ephemeren Znodes daher bestehen.

Erst wenn der Client nicht innerhalb des ausgehandelten Session-Timeouts zurückkehrt, läuft die Session ab. Zookeeper entfernt dann deren ephemere Znodes. Kommt der Client später zurück, beginnt er eine neue Session und muß seine ephemeren Znodes neu anlegen.

Ein Client kann außerdem Watches auf Znodes setzen. Ein Watch ist eine einmalige Benachrichtigung darüber, daß sich der beobachtete Zustand geändert haben könnte. Danach muß der Client den Watch erneut installieren. Die Benachrichtigung ist kein vollständiger Diff und keine dauerhaft gespeicherte Event-History, sondern nur eine Mitteiling, daß sich die Dinge geändert haben und die Znodes unter dem Watch neu gelesen werden müssen. Mehrere Änderungen können zu einer einzigen relevanten Benachrichtigung zusammenfallen (“Debouncing”).

Diese vier Eigenschaften ergeben zusammen das Discovery-Primitive:

  • Der Znode-Baum organisiert Services und Pools.
  • Persistente Znodes definieren deren Struktur.
  • Ephemere Znodes repräsentieren die an eine Session gebundenen Mitglieder.
  • Watches invalidieren die bei den Consumers gespeicherte Sicht.

Zookeeper machte Membership explizit

In Zookeeper konnte jeder Pool als Pfad und jeder geeignete Server als dessen Kind dargestellt werden. Änderungen wurden damit explizite Zustandsänderungen statt indirekte Folgen von DNS-Caches.

flowchart TD
P["/production-readers-rz-a<br/>persistent"]
A["db-rza-017<br/>ephemeral<br/>10.1.17.12:3306"]
B["db-rza-023<br/>ephemeral<br/>10.1.23.12:3306"]
C["db-rza-041<br/>ephemeral<br/>10.1.41.12:3306"]
P --> A
P --> B
P --> C

Der Pool-Knoten ist persistent. Die Member sind ephemeral und gehören jeweils einer Zookeeper-Session. Endet die Session, entfernt Zookeeper die zugehörigen Member automatisch.

Auf einem MySQL-Server zum Beispiel läuft ein Healthchecker neben dem mysqld, verbindet sich mit diesem und entscheidet, ob dieser Dienst bereit ist. Ist er bereit, erzeugt der Healthchecker den ephemeren Knoten. Ist er nicht mehr geeignet, entfernt er ihn oder beendet seine Session.

Ein ephemerer Knoten ist dabei ein Healthcheck-Resultat, aber keine Garantie. Er bedeutet:

Der Besitzer dieser Zookeeper-Session lebt und hält den Endpoint weiterhin für geeignet.

Ein abgestürzter Server bleibt daher bis zum Ablauf der Session sichtbar. Ein hängender Healthchecker kann einen ungeeigneten Server weiterhin veröffentlichen. Der Client muß Verbindungsfehler trotz Service Discovery behandeln können.

Watches sind keine Events

Zookeeper-Watches sind keine zuverlässige Ereignisaufzeichnung. Ein Watch bedeutet nicht: „Verarbeite genau diese Änderung.“ Er bedeutet:

Der beobachtete Zustand könnte sich geändert haben. Lies ihn neu.

Das ist für Discovery ideal. Der Client interessiert sich nicht für jede Zwischenstufe einer Änderung:

db17 hinzugefügt
db23 entfernt
db17 geändert
db41 hinzugefügt

Er braucht nur den Zustand, gegen den er schließlich konvergieren soll:

Der Pool besteht jetzt aus db17 und db41.

Der Consumer installiert seine Watches erneut und liest den vollständigen Pool. Wird währenddessen wieder eine Änderung signalisiert, beginnt er den Vorgang noch einmal. Nach einem Verbindungsabbruch macht er ebenfalls einen vollständigen Rescan.

Verpaßte Zwischenzustände sind bedeutungslos. Es muß nur irgendwann wieder eine vollständige Reconciliation stattfinden. Das System garantiert Eventual Convergence, nicht die Beobachtung jedes Updates.

flowchart TD
S["Service und Healthchecker"] -->|"ephemerer Member"| Z["Zookeeper Pool"]
Z -->|"Watch: Zustand könnte geändert sein"| A["lokaler Agent"]
A -->|"vollständigen Pool lesen"| Z
A -->|"temporäre Datei + rename(2)"| F["lokale Endpoint-Datei"]
F -->|"bei Verbindungsaufbau lesen"| C["Anwendung"]
C -->|"direkte Verbindung"| S

Änderungen dürfen zusammenfallen

Bei einer größeren Wartungsaktion können viele Pool-Änderungen kurz hintereinander auftreten. Es ist nicht sinnvoll, jeden Zwischenzustand auf alle Clients zu verteilen.

Der Agent wartet nach einer Invalidierung eine zufällige Zeit bis zu einem konfigurierten Maximum. Weitere Änderungen in dieser Zeit werden zusammengefaßt. Nach dem Ende des Änderungsbursts erzeugt der Agent eine neue vollständige Sicht.

Das hat zwei erwünschte Effekte:

  • Viele Änderungen erzeugen nur wenige neue Dateien.
  • Nicht alle Agents lesen gleichzeitig denselben Pool aus Zookeeper.

Der zufällige Delay begrenzt nicht die Zeit seit der ersten Änderung. Bei andauernden Änderungen kann die Materialisierung weiter aufgeschoben werden. Das ist hier akzeptabel: Eine konsistente Sicht nach Ende des Bursts ist wichtiger als die Wiedergabe aller Zwischenzustände.

Die Datei ist die lokale materialisierte Sicht

Der Agent schreibt für den Pool eine einfache Datei:

10.1.17.12:3306 # db-rza-017
10.1.23.12:3306 # db-rza-023
10.1.41.12:3306 # db-rza-041

Das Format ist nicht wesentlich. Es kann Text, JSON oder ein Format für eine bestimmte Clientbibliothek sein. Wesentlich ist, daß die Datei eine vollständige materialisierte Sicht darstellt.

Die Anwendung muß dadurch keine Zookeeper-Bibliothek enthalten. Sie braucht keine Sessions zu verwalten, keine Watches neu zu installieren und keine Reconnects zu implementieren. Beim Aufbau einer Verbindung liest sie die lokale Datei, wählt einen passenden (zufälligen!) Endpoint und verbindet sich direkt.

Die Datei ist gleichzeitig ein Read Cache. Ein Verbindungsaufbau erzeugt keinen Read im Zookeeper-Cluster. Der Agent liest nur beim Start, nach einer Invalidierung und nach einem Reconnect. Ein Agent pro Host kann dieselbe Sicht für viele lokale Prozesse bereitstellen.

Die Datei wird atomar ersetzt

Eine Serverliste darf beim Lesen nicht halb alt und halb neu sein. Der Agent schreibt den neuen Inhalt deshalb nicht direkt in die Zieldatei:

endpoints.$pid schreiben
endpoints.$pid schließen
rename("endpoints.$pid", "endpoints")

Die temporäre Datei liegt im selben Verzeichnis und damit auf demselben Filesystem wie das Ziel. rename(2) ersetzt dann den Verzeichniseintrag in einer atomaren Operation. Ein Reader sieht entweder die alte oder die neue vollständige Datei, niemals eine teilweise geschriebene Version.

Das Verfahren wird ausführlicher in But is it atomic? diskutiert.

Reader müssen die Datei für eine neue Sicht erneut öffnen. Ein tail -f kann am alten Inode hängenbleiben, nachdem der Name bereits auf die neue Datei zeigt. Darum verwenden wir für solche Dateien immer tail -F: Es folgt dem Dateinamen, erkennt den Austausch und öffnet die neue Datei.

Für diese Anwendung ist atomare Sichtbarkeit wichtig, nicht garantierte Persistenz nach einem Stromausfall. Die Datei ist ein rekonstruierbarer Cache, kein autoritativer Datenspeicher. Zusätzliche fsync()-Operationen lösen hier kein relevantes Discovery-Problem.

Loss of Control ist nicht Loss of Service

Zookeeper ist ein Konsenssystem. Wenn es kein Quorum herstellen kann, darf es vorübergehend lieber keine Antwort geben als eine möglicherweise falsche. Das ist für die Control Plane die richtige Entscheidung, darf aber nicht jeden Verbindungsaufbau in der Data Plane blockieren.

Der Agent behält bei einem Zookeeper-Ausfall die zuletzt vollständig geschriebene Datei. Bestehende und neue Prozesse können weiter Endpoints auswählen. Nach der Rückkehr von Zookeeper liest der Agent den gesamten Pool neu und konvergiert gegen den aktuellen Zustand.

Die Materialisierung erhält also die Verfügbarkeit der Data Plane auf Kosten der Aktualität:

Eigenschaft Zookeeper Lokale Datei
Zustand aktuell und koordiniert möglicherweise veraltet
Verfügbarkeit bei fehlendem Quorum eingeschränkt lokal weiter lesbar
Zugriff Netzwerk und Clientbibliothek lokaler Dateizugriff
Änderung Watch als Invalidierung atomarer Austausch
Last Read bei Reconciliation beliebig viele lokale Reads
Aufgabe Control Plane materialisierte Data-Plane-Konfiguration

Die Datei garantiert während eines Ausfalls nicht, daß jeder gelistete Server noch erreichbar ist. Sie garantiert eine vollständige letzte Auswahl. Der Client probiert beim Verbindungsfehler einen anderen Endpoint. Aus einem Ausfall der Control Plane wird dadurch Staleness statt eines unmittelbaren Ausfalls der Data Plane.

Dieses Verhältnis zwischen Control Plane und Data Plane ist aus zwei anderen Perspektiven in Service Mesh und Konsenssysteme beschrieben.

Was das Pattern garantiert

Das System gibt drei nützliche Garantien:

  • Ein Reader sieht eine vollständige alte oder neue Endpoint-Datei.
  • Der letzte bekannte Zustand bleibt ohne verfügbare Control Plane nutzbar.
  • Nach der Rückkehr der Control Plane konvergiert der Agent gegen den aktuellen Pool.

Es gibt bewußt keine Garantie für Folgendes:

  • Der Consumer sieht jeden Zwischenzustand.
  • Die lokale Datei ist während eines Ausfalls aktuell.
  • Ein gelisteter Endpoint ist beim nächsten Connect tatsächlich erreichbar.
  • Alle Consumers wechseln gleichzeitig auf denselben neuen Zustand.

Diese Einschränkungen sind keine Fehler im Design. Sie sind die Folge einer bewußten Entscheidung: Service Discovery muß für diesen Anwendungsfall schnell und betriebssicher konvergieren, aber sie muß kein synchrones Protokoll im Request-Pfad sein.

Das allgemeine Pattern

MySQL war der konkrete Anwendungsfall, ist aber keine Voraussetzung. Das Pattern paßt, wenn folgende Bedingungen gelten:

  • Es gibt dynamische Pools gleichartiger Endpoints.
  • Ein lokaler Agent kann den zentralen Zustand beobachten.
  • Consumers können mit einer vollständigen, kurzzeitig veralteten Sicht leben.
  • Consumers behandeln Fehler einzelner Endpoints ohnehin selbst.
  • Der zentrale Discovery-Dienst soll nicht im Request- oder Connect-Pfad liegen.

Eine ausführbare Python-Demonstration steht im Repository zkdemo . Sie registriert ephemere Server, beobachtet Pool-Änderungen, faßt Änderungsbursts zusammen und ersetzt die lokale Endpoint-Datei atomar.

Ich behaupte nicht, daß eine Datei moderner als DNS oder ein Load-Balancer wäre. Aber wir sehen hier eine saubere Trennung: Zookeeper entscheidet konsistent über Membership. Der Agent materialisiert diese Entscheidung lokal. Die Anwendung arbeitet mit dem letzten vollständigen Ergebnis weiter, auch wenn die Control Plane gerade nicht erreichbar ist.

Top

Valuable News – 2026/08/17

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

[CFT] FreeIPA Server on FreeBSD.
https://github.com/joneum/FreeBSD-freeipa-server

Sharing Dual Licensed Drivers between Linux and FreeBSD.
https://freebsdfoundation.org/blog/sharing-dual-licensed-drivers-between-linux-and-freebsd/

FreeBSD Git Weekly: 2026-08-03 to 2026-08-09.
https://freebsd-git-weekly.tarsnap.net/2026-08-03.html

OpenSSH 10.5 Released.
https://undeadly.org/cgi?action=article;sid=20260811113606

FreeBSoD: Leveraging Language Models to Find and Exploit Kernel Bugs (Part 1 of 2).
https://praetorian.com/blog/ai-vulnerability-research-freebsd-kernel/

FreeBSoD: Leveraging Language Models to Find and Exploit Kernel Bugs (Part 2 of 2).
https://praetorian.com/blog/llm-kernel-exploit-development/

First Successful FreeIPA Server Installation on FreeBSD.
https://linkedin.com/posts/jochen-neumeister-361096147_freebsd-freeipa-share-7492972762148515840-0rvx/

FreeBSD Laptop Compatibility.
https://freebsdfoundation.github.io/freebsd-laptop-testing/

DTrace Memory Analyzer for FreeBSD.
https://github.com/oshogbo/dtrace-memanalyzer

LibreFS (Minio Fork) Builds and Runs Well on FreeBSD.
https://x.com/vermaden/status/2087284456889397311

New video(4) Driver Landed in CURRENT.
https://lists.freebsd.org/archives/freebsd-current/2026-August/010629.html

Skip2 Networks – What is FreeBSD?
https://www.skip2.net/blog/what-is-freebsd

Move FreeBSD System Between ZFS Disks.
https://vermaden.wordpress.com/2026/08/14/move-freebsd-between-zfs-disks/

Custom FreeBSD UCARP Setup.
https://vermaden.wordpress.com/2026/08/15/custom-freebsd-ucarp-setup/

Running Wiki.js in FreeBSD 15.1 Jail.
https://dkade.com/posts/wikijs_freebsd_jail/

FreeBSD 14.5-BETA2 Now Available.
https://lists.freebsd.org/archives/freebsd-stable/2026-August/004277.html

Minimal Intel ME Diagnostic Driver for FreeBSD in Review.
https://reviews.freebsd.org/D58863

Incoming www/linux-brave DRM ready Port Comming to FreeBSD.
https://linkedin.com/posts/dteske_i-have-recently-put-a-fair-amount-of-work-ugcPost-7494197143814017024-5VZ1/

There is No Linux Admin.
https://vivianvoss.net/blog/no-linux-admin

What 'sh -x' Mostly Does Not Tell You About Shell Script.
https://utcc.utoronto.ca/~cks/space/blog/programming/ShXWhatYouMiss

SeaweedFS on FreeBSD and Rather Deep Rabbit Hole.
https://tara.sh/posts/2026/2026-08-13_seaweedfs_freebsd_rabbithole/

Reproduce Missing etcmerge(8) from FreeBSD 15.0 to 15.1 Upgrade.
https://dan.langille.org/2026/08/12/attempting-to-reproduce-the-missing-etcmerge-from-15-0-to-15-1-upgrade/

FreeBSD 14.3 on Parallels 26.
https://conradresearch.com/design-distilled/freebsd-14-on-parallels-26

Cloudflare Tunnels on FreeBSD 14.3.
https://conradresearch.com/design-distilled/cloudflare-tunnels-on-freebsd

Installing FreeBSD on OVHcloud Bare Metal Server.
https://conradresearch.com/design-distilled/installing-freebsd-on-OVHcloud-bare-metal-server

Installing Infisical on FreeBSD.
https://conradresearch.com/design-distilled/installing-infisical-on-freebsd

Building Turso Golang Client on FreeBSD.
https://conradresearch.com/design-distilled/building-turso-golang-client-on-freebsd

Temporal Dev Server on FreeBSD.
https://conradresearch.com/design-distilled/temporal-dev-server-on-freebsd

Immutable Software Deploys Using ZFS Jails on FreeBSD.
https://conradresearch.com/design-distilled/immutable-software-deploy-zfs-jails

Tailscale Exit Node on FreeBSD.
https://conradresearch.com/design-distilled/tailscale-exit-node-on-freebsd

Tailscale SSH Server Access on FreeBSD.
https://conradresearch.com/design-distilled/tailscale-ssh-server-access-on-freebsd

The fw_update(8) Utility Explained and How It Works.
https://bsd-audit.com/openbsd/fw_update-explained/

Replacing security/pam-ssh-agent-auth with security/pam-ssh-agent Package.
https://dan.langille.org/2026/08/17/replacing-security-pam-ssh-agent-auth-with-security-pam-ssh-agent/

FreeBSD Home Server – Part 1 – OS Install.
https://subnetspider.com/2026/08/16/freebsd-home-server-part1.html

FreeBSD Wochenruckblick 2026/07/6-12. [German]
https://tgeppert.de/2026/07/13/freebsd-wochenrueckblick-6-12-juli-2026/

FreeBSD Wochenruckblick 2026/07/14–20. [German]
https://tgeppert.de/2026/07/20/freebsd-wochenrueckblick-14-20-juli-2026/

UNIX/Audio/Video

NetBSD 11 Released with Continued i386 and Improved RISC-V Support.
https://youtube.com/watch?v=H_rOjsGDM7g

Install FreeBSD as Home Lab Server.
https://youtube.com/watch?v=EEkfeibYirE

Configure FreeBSD as Home Lab Server.
https://youtube.com/watch?v=2jgiIR9IAZQ

Configure and Use VNET Jails in FreeBSD.
https://youtube.com/watch?v=IdMloTbM9lE

Your Home Lab FreeBSD Bhyve Virtualization Host.
https://youtube.com/watch?v=PQ-mo–2pxg

Create Jail in Sylve – FreeBSD Virtualization and Container GUI – 2026 Version.
https://youtube.com/watch?v=LiFn3hF4UPs

BVCP Bhyve GUI Control Panel for FreeBSD.
https://youtube.com/watch?v=fo3HNFNS4-Y

BSD Now 676: 10 PRINT GOTO 10.
https://www.bsdnow.tv/676

Hardware

Keep Android Open – Stop Google from Limiting APK File Usage.
https://change.org/p/keep-android-open-stop-google-from-limiting-apk-file-usage

Trezor Crypto Wallet Confirms 13,000 Customers Details Exposed in Breach.
https://theregister.com/security/2026/08/14/trezor-confirms-13000-customers-details-exposed/5287734

Supersonic Trebuchet.
https://youtube.com/watch?v=Co57SfcT-h0

Were Mac Touch Bar Problems Software Rather Than Hardware?
https://unsung.aresluna.org/deeper-dive-were-touch-bars-problems-software-rather-than-hardware/

Selection of Older But Still Relevant Tests (Part 5).
https://hwcooling.net/en/selection-of-older-but-still-relevant-tests-part-5/

Life

Other

Gotosocial Reverse Proxy with Wireguard.
https://owleyes.blue/posts/gotosocial-reverse-proxy-with-wireguard/

Desktop Calendar – Thunderbird Design Journey.
https://blog.thunderbird.net/2026/08/desktop-calendar-a-design-journey/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Replacing security/pam-ssh-agent-auth with security/pam_ssh_agent

Post by Dan Langille via Dan Langille's Other Diary »

Yesterday, I learned that security/pam-ssh-agent-auth is abandoned and deprecated.

Looking back, I first used this about 13 years ago, and wrote about it while describing how I set up ansible clients.

Today, I tried one of the two recommended replacement:

  1. security/pam_rssh – mentioned by Romain Tartière
  2. security/pam_ssh_agent – the one I’m trying

Note the package vs the port

Note this point. The package name is:

[15:03 empty dvl ~] % pkg info -x pam-ssh-agent
pam-ssh-agent-0.9.7

The origin / port name is:

[15:03 empty dvl ~] % pkg info -o pam-ssh-agent
pam-ssh-agent-0.9.7            security/pam_ssh_agent

Note dashes vs underscores. I original had the origin wrong in this title.

Very much a drop-in replacement

Here’s what I did.

Install it:

The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	pam-ssh-agent: 0.9.7 [local]

Number of packages to be installed: 1

The process will require 1 MiB more space.
413 KiB to be downloaded.

Proceed with this action? [y/N]: y
[empty.int.unixathome.org] [1/1] Fetching pam-ssh-agent-0.9.7: 100%   413 KiB 422.5 kB/s    00:01    
Checking integrity... done (0 conflicting)
[empty.int.unixathome.org] [1/1] Installing pam-ssh-agent-0.9.7...
[empty.int.unixathome.org] [1/1] Extracting pam-ssh-agent-0.9.7: 100%

What did it install?

[11:32 empty dvl ~] % pkg info -l pam-ssh-agent
pam-ssh-agent-0.9.7:
	/usr/local/lib/pam_ssh_agent.so

Adjust /usr/local/etc/pam.d/sudo:

#auth sufficient /usr/local/lib/pam_ssh_agent_auth.so file=~/.ssh/authorized_keys
auth sufficient /usr/local/lib/pam_ssh_agent.so      file=~/.ssh/authorized_keys
auth required pam_deny.so
account include system
session required pam_permit.so

The commented out line is the old. The new line is the second line.

That’s it. Nothing to restart…

However, there is more to this configuration than shown. That did not need to be modified for this. See the “Configure ssh agent auth for sudo” section of this post.

I did delete the deprecated package: sudo pkg delete pam_ssh_agent_auth

Top

DFG Heisenberg project post-mortem

Post by Andreas K. Hüttel via the dilfridge blog »

This post is an excerpt from the formally non-public part of my DFG Heisenberg grant final report (about half of the actually interesting part). I hope it will help people do better. The remaining part of the report can be found in a subsequent blog post. Some personal information has been redacted.

The backstory

In 2018, when I first submitted a Heisenberg program proposal, funding was declined. One referee report was full of misinformation and misinterpretations, the other criticized the methods in one of the suggested research subtopics. Overall this sufficiently lowered the ranking of the entire submission. Anyway, as the saying goes, stand up again, brush down coat, re-adjust crown, keep going. The proposal was updated, the doubtful part replaced, a response to the misinformation added, and of course also some more exciting research ideas included.

At that point I decided to go for full risk. While the DFG Emmy Noether program deliberately funds a junior research group consisting of a PI and PhD students or a post-doc, the DFG Heisenberg program is more adjusted along the customs of humanities and funds the PI alone. It is still supposed to provide a base for independent research at the level of associate professor though, which is why one can concurrently submit a supporting research grant proposal for equipment and personnel. The advantage of doing so is that this can form a coherent overall funding package, the disadvantage is that a negative review of any part of the package will drag it down in its entirety, see above.

Complementing the overarching Heisenberg proposal for my own position and its research, I submitted in 2019 two additional research proposals ("Einzelanträge"). One focused on the continuation of the Emmy Noether project, tuning optomechanics of single-wall carbon nanotubes towards strong coupling and coherent control, and including such nice ideas as, e.g., coupling mechanics with coherent states in double quantum dots. This was a highly complex project, intended for two PhD students (and the two students were really required because of the combination of multi-step fabrication and complicated experiment). In addition, it was adjusted to fit to the topic of a Graduate Research School (GRK) proposal under preparation back then in Regensburg. The idea of the second grant was to try out something new, and establish quantum transport measurements on MoS2 nanotubes – a material where already a lot of optical measurements existed but the transport physics of quantum dots was so far completely unexplored. Here, one PhD student was requested; furthermore the topic was deliberately chosen to be in the area of interest of a new Regensburg Collaborative Research Centre (SFB), SFB 1277, in the hope of further financial support options.

Because of the significant amount of university-bound equipment acquired from SFB funds, and the potential difficulties of moving “my” large dilution refrigerator, I chose again Universität Regensburg as host institution.

In the meantime, my employment contract in Regensburg ran out (thanks Wissenschaftszeitvertragsgesetz), so I went to the Low Temperature Laboratory, Department of Applied Physics, Aalto University, Finland for one year as full-time employed visiting professor; my thanks go to Prof. Pertti Hakonen for making this possible. Two weeks later COVID broke out, but Aalto was a great place to be both scientifically and to sit out the pandemic. And in autumn a sequence of excellent news followed; the Heisenberg grant was approved, and in addition the Emmy Noether work on microwave optomechanics was awarded the Walter Schottky Prize 2021 of the German Physical Society. So, things were clearly brightening up, or so I thought.

A more detailed inspection of the grant approval letter provided a somewhat more mixed image. One referee explicitly and clearly supported the request for two PhD students in the optomechanics project, the other also explicitly lauded all details, including the excellent funding plan, of the optomechanics project, but additionally stated that the impact of the MoS2 nanotube project would be larger (this was likely written before the announcement of the Walter Schottky Prize). As result, only one PhD position for optomechanics was granted by the funding committee. Hope always dies last, but in retrospect I can now confirm my immediate suspicion that this reduction of funding killed the optomechanics project from the start. My initial “plan B” for additional optomechanics funds was not available anymore, since the Regensburg Graduate Research School had in the meantime made an ultrafast turn towards other research topics. Further, even though this project was the direct continuation of the Walter Schottky Prize work, it turned out to be extremely difficult to find and hire a PhD student for it. The project started on 16 March 2021, and only on 1 August 2022 a PhD student arrived.

In comparison, the MoS2 nanotube project start-up went much more smooth, and [...] started work on his PhD straight on the 16 March 2021. All hope of a financially significant participation in the Regensburg SFB 1277 was however shattered already by a brief conversation with the back then SFB speaker, who made clear that nothing beyond appointing me “associated member” would even be considered. Well, you can't allow “junior scientists” to become too successful…

Scientific progress

I had attempted to keep the optomechanics project going in Regensburg during my time in Finland via a remotely-supervised MSc student, who successfully optimized coplanar waveguide resonator geometries and produced and tested the corresponding devices. When I came back, I quickly found another MSc student who was very enthusiastic to start with optomechanics experiments. However, we also quickly found out that the nanotube growth oven had broken in the meantime, requiring the whole process to be optimized from the start, and the MSc project literally became a year of getting carbon nanotube growth going again from zero. Now in 2026 (!) the quality of nanotubes transferred into a circuit is finally showing excellent results again. That said, the nanotube optomechanics project had many delicate parts, from nanotube growth and transfer to coplanar resonator chip design and fabrication, and no number of MSc students recruited into my group could really replace the missing second PhD student.

[...] On the MoS2 nanotube side, progress was slow but steady, and the PhD student did excellent work. Making contacts to MoS2 nanotubes turned out to be even more complex than contacts to carbon nanotubes or a 2D MoS2 monolayer. A technical breakthrough in the latter system by researchers from MIT and TSMC, among others, provided a path forward, and indeed their approach also led to occasional good results with the MoS2 nanotubes. Obtaining these good results reproducibly, however, was again another complex optimization step; we solved that in 2025 and subsequently managed first physically interesting low-temperature measurements.

In general, across both subprojects, work was slowed down very much by continuous equipment break-downs and oddities in the Regensburg cleanroom. "The SEM for e-beam writing is down" turned out to be one highly regular e-mail subject (for any possible value of "the SEM"). Mystery changes in resist properties, micrometer-scale shifts in the written structures, interruptions in the air conditioning that led to water condensation in the whole cleanroom, ... The department bought a Heidelberg Instruments mask-less aligner (a laser writer for lithography), which was nice but of limited usefulness – since for nanophysics you actually need nano-resolution! Plus there were some other annoying events; e.g., during the installation of a new dilution refrigerator next door someone opened up our (then evacuated) 3He/4He circuit in the pump room to air, which we only noticed when we tried to cool down and suddenly were pumping air into the cold dilution refrigerator insert. Luckily, nearly no isotope mixture was lost.

Experimental work took significantly longer than expected, but eventually did yield interesting results right at the end. As stated above, for the research details of the two subprojects, I refer to the final reports of the research grants [...] and [...].

[...]

Early career stage researchers

I am proud to be able to say that during my ~16 years in Regensburg I have supervised 7 PhD students and 27 MSc or Diplom students. During the Heisenberg period specifically, two PhD students should be named, [...] and [...]. [...] is currently helping a new professor in Regensburg build up his lab on a post-doc position and considering remaining in academia (which he would definitely be suited for). [...] is still busy measuring beautiful data and writing up his dissertation, while being paid by SFB1277.

At the end of the official project runtime (and my employment) in March three MSc students were still active; two have graduated by now, the third one recently handed in his thesis.

Top

DFG Heisenberg post-mortem, part 2

Post by Andreas K. Hüttel via the dilfridge blog »

This post contains remaining parts of my DFG Heisenberg grant final report, by now long submitted and accepted by the DFG. I hope it will be of use, in particular to future DFG grant candidates. The text also somewhat explains why I'm fed up with academia, Regensburg, and physics right now (in no particular order and to varying degrees). The first part of the report can be found in a previous post. Some personal information has been redacted.

Qualification, career, …

During the Heisenberg period 2021-2026, submitted applications for professorships resulted in 8 invitations for presentation and discussion: [...]

A recurrent topic over all my years in academia was that appointment procedures for professorships either failed or that positions were handed to local Emmy Noether, Heisenberg, Juniorprofessor, etc. candidates – much more than expected from past times when the spectre of unwanted “Hausberufung” (in-house appointment) was looming over such events. From that point of view it was equally disappointing that, once I applied for a position in Regensburg, I was immediately told by a member of the appointment commission that my application would be sorted out for exactly that reason. When I spoke to an external member of the appointment commission for that professorship, that external member did not even know that I had applied for the position. In parallel, somewhat doubtful direct appointments to professorships took place in the department.

Further frustration originated from the fact that the Regensburg physics department decided to rewrite their internal regulations such that even after 6 years of Privatdozentur (senior lecturer with habilitation) they only award the title of Außerordentlicher Professor (associated professor, a title without salary) to candidates with a permanent university position. My corresponding petition never received any written response, only oral comment that this is not done anymore. The motivation behind all this is unclear, but it certainly does not make Regensburg more attractive to (somewhat) junior scientists.

A four-eyes career conversation with Mr. Dean of the Faculty resulted in charming comments such as “the best we can do is half an E13 position in the experimental collection maintenance of the undergraduate lectures in a year or so, maybe you could do some research on the side there” and “as a Heisenberg you are an academic legacy problem” (the original German term was “akademische Altlast”). Plus remarks like “we are not going to hand out any (even unpaid) teaching assignments (“Lehraufträge”) to an external Privatdozent anymore, so you’ll lose that title anyway soon for lack of teaching.”

The final cherry on top was then getting berated by another influential senior faculty member about producing insufficient results and being unable to supervise – which is pretty thick coming from someone who barely finds any MSc students, having a reputation among students for erratic grading there. To quote a colleague from Regensburg with permanent employment who should probably stay anonymous, “Der professorale Dünkel ist stark in der Fakultät” (roughly translated, the professorial sense of self-importance is tantamount in the department).

With all these experiences as background, I strongly recommend to anyone with a grant such as DFG Emmy Noether or Heisenberg not to join the Regensburg physics department, but go somewhere else. You and your work will be valued more, and that by far compensates for any Regensburg “excellence”. [...]

Teaching and scientific events

With already significant teaching experience, I decided to mostly focus on research and supervision during the Heisenberg period – with the exception of a “Low Temperature Physics” course where the usual lecturer was on sabbatical. Regarding scientific events, I had requested a moderate amount of funding for a workshop on inorganic nanotubes in [...], which would have fitted quite well as add-on to the workshops of SFB 1277. This was however declined – accordingly, it also did not have priority for me anymore either.

Perspective

I enjoyed university research and teaching as well as working with students, establishing a research group, quantum device fabrication, and complex measurements very much. With the “orderly shutdown” of the group comes that still some students need to graduate and that some results will still be written up and submitted for publication (at the time of this report two papers have been submitted and another two or three might follow); however, I have cleared out my office the day of the end of my employment contract. 

Out of a certain stubbornness, I will keep teaching my 2SWS per year Titellehre for now – as confirmed by the faculty administration, a teaching appointment (“Lehrauftrag”) is legally not required for that, and as a consequence the students will be treated to a block course “How to write a thesis, present a talk, …” in October. However, after over 15 years in Regensburg I have learned that being nice, friendly, cooperative, and helpful might get you friendly smiles and occasional money there but not any sustainable support, and that the Regensburg physics system prefers “junior scientists” to be subordinates, not colleagues. Of course someone who does not have the visible backing of his current university immediately also has a weaker position when applying elsewhere. 

There’s a lot more I could write, but that would go beyond scope and purpose of this report. Anyway, it is now time to make a bonfire out of lecture notes, to go out into the wide world, and to do something entirely new. Which, barring big surprises, most likely will not involve physics or a university anymore. 

Thanks to Duong Thi Hai Yen on vecteezy.com for the tomato.

Top

Move FreeBSD System Between ZFS Disks

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

I made a funny mistake in one of my clients when installing FreeBSD 15.1-RELEASE on their server … I have chosen wrong disk for the installation. The server had 4 local disks – 2 of them were some really low end SATA SSD drives (ada0 and ada1) and the server also had 2 fast NVMe disks (nda0 and nda1). I used Auto (ZFS) option in the bsdinstall(8) installer and chosen (wrongly) the ada1 disk.

This is how my ZFS pool looked like after installation.

freebsd # zpool status zroot
  pool: zroot
 state: ONLINE
config:

	NAME        STATE     READ WRITE CKSUM
	zroot       ONLINE       0     0     0
	  ada1p4    ONLINE       0     0     0

… but this is the ZFS world – nothing stops You from adding as many drives to ZFS mirror as You want – so for a start I used FreeBSD gpart(8) tool to copy the partitions from ada1 disk to the NVME nda0 and nda1 drives.

freebsd # gpart backup ada1 | gpart restore nda0

freebsd # gpart backup ada1 | gpart restore nda1

Then as FreeBSD was installed with BIOS+UEFI option I cloned the freebsd-boot partition and the UEFI partition.

freebsd # dd bs=1m if=/dev/ada1p1 of=/dev/nda0p1 status=progress

freebsd # dd bs=1m if=/dev/ada1p1 of=/dev/nda1p1 status=progress

freebsd # dd bs=1m if=/dev/ada1p2 of=/dev/nda0p2 status=progress

freebsd # dd bs=1m if=/dev/ada1p2 of=/dev/nda1p2 status=progress

Next I added these nda0 and nda1 as mirror devices for ada1 disk. For the record – the ada1p3 is a SWAP device – no need to clone it for obvious reasons.

freebsd # zpool attach zroot ada1p4 nda0p4

freebsd # zpool attach zroot ada1p4 nda1p4

This is when ZFS decided to do the resilvering process immediately (which is good).

freebsd # zpool status zroot
  pool: zroot
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Aug 13 18:47:35 2026
	199G / 199G scanned, 15.5G / 199G issued at 428M/s
	31.0G resilvered, 7.76% done, 00:07:20 to go
config:

	NAME        STATE     READ WRITE CKSUM
	zroot       ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    ada1p4  ONLINE       0     0     0
	    nda0p4  ONLINE       0     0     0  (resilvering)
	    nda1p4  ONLINE       0     0     0  (resilvering)

errors: No known data errors

We can even track the status of that replication.

freebsd # zpool iostat -v zroot 5
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zroot        199G   229G     25    758  3.11M  42.6M
  mirror-0   199G   229G  3.21K  95.9K   402M  5.38G
    ada1p4      -      -     25    693  3.11M  36.5M
    nda0p4      -      -      0  4.15K    281   403M
    nda1p4      -      -      0  4.41K    305   413M
----------  -----  -----  -----  -----  -----  -----
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zroot        199G   229G  2.89K  8.29K   369M   850M
  mirror-0   199G   229G  2.89K  8.29K   369M   850M
    ada1p4      -      -  2.89K    383   369M  37.1M
    nda0p4      -      -      0  4.00K      0   406M
    nda1p4      -      -      0  3.91K      0   406M
----------  -----  -----  -----  -----  -----  -----
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zroot        199G   229G  2.99K  7.30K   378M   827M
  mirror-0   199G   229G  2.99K  7.30K   378M   827M
    ada1p4      -      -  2.99K    185   378M  19.9M
    nda0p4      -      -      0  3.54K  7.20K   404M
    nda1p4      -      -      0  3.58K      0   404M
----------  -----  -----  -----  -----  -----  -----

After resilvering process is finished we can detach the ada1 drive from the ZFS mirror.

freebsd # zpool detach zroot ada1p4

… and after all these quite simple and predictable operations – everything when the system was still running – we have FreeBSD installation migrated from ada1 disk to ZFS mirror on two NVMe nda0 and nda1 disks as shown below.

freebsd # zpool status zroot
  pool: zroot
 state: ONLINE
  scan: scrub repaired 0B in 00:00:08 with 0 errors on Thu Aug 13 23:20:15
2026
config:

	NAME        STATE     READ WRITE CKSUM
	zroot       ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    nda0p4  ONLINE       0     0     0
	    nda1p4  ONLINE       0     0     0

errors: No known data errors

Its kinda short article (for my standards) but it shows what was needed and possible with ZFS and other FreeBSD tools to do the job. One more thing that its needed to be done is that in BIOS/UEFI you need to add boot option specifying the path to the loader.efi file – separate one for each NVMe drive.

UPDATE 1 – BIOS/UEFI Boot Option

Someone asked me how this is exactly done … and it is done differently in each BIOS/UEFI manufacturer – below one is from some Lenovo ThinkServer system server.

First enter into BIOS/UEFI and go to Boot Settings part.

Next select Add UEFI Full Path Boot Option there.

In the next ‘window’ use Select Device Path Option position.

Next select the drive You know You installed FreeBSD on – this is one of the two NVMe drives I moved FreeBSD to.

Next click <efi> dir.

Next click <freebsd> dir.

Next select loader.efi file.

Next add wanted name – I used FreeBSD/nda0 there.

Click Commit Changes and Exit option.

Next click Change Boot Order and set that FreeBSD/nda0 option as first one.

Do the same for another NVMe disk and add another FreeBSD/nda1 option there.

Alternatively You can do that from within FreeBSD system using efibootmgr(8) command – Important efibootmgr(8) Command – details here.

 

EOF

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Top