Planet

Last updated: 2026-01-14 06:45:01 UTC

Patch Tuesday, January 2026 Edition

Post by Brian Krebs via Krebs on Security »

Microsoft today issued patches to plug at least 113 security holes in its various Windows operating systems and supported software. Eight of the vulnerabilities earned Microsoft’s most-dire “critical” rating, and the company warns that attackers are already exploiting one of the bugs fixed today.

January’s Microsoft zero-day flaw — CVE-2026-20805 — is brought to us by a flaw in the Desktop Window Manager (DWM), a key component of Windows that organizes windows on a user’s screen. Kev Breen, senior director of cyber threat research at Immersive, said despite awarding CVE-2026-20805 a middling CVSS score of 5.5, Microsoft has confirmed its active exploitation in the wild, indicating that threat actors are already leveraging this flaw against organizations.

Breen said vulnerabilities of this kind are commonly used to undermine Address Space Layout Randomization (ASLR), a core operating system security control designed to protect against buffer overflows and other memory-manipulation exploits.

“By revealing where code resides in memory, this vulnerability can be chained with a separate code execution flaw, transforming a complex and unreliable exploit into a practical and repeatable attack,” Breen said. “Microsoft has not disclosed which additional components may be involved in such an exploit chain, significantly limiting defenders’ ability to proactively threat hunt for related activity. As a result, rapid patching currently remains the only effective mitigation.”

Chris Goettl, vice president of product management at Ivanti, observed that CVE-2026-20805 affects all currently supported and extended security update supported versions of the Windows OS. Goettl said it would be a mistake to dismiss the severity of this flaw based on its “Important” rating and relatively low CVSS score.

“A risk-based prioritization methodology warrants treating this vulnerability as a higher severity than the vendor rating or CVSS score assigned,” he said.

Among the critical flaws patched this month are two Microsoft Office remote code execution bugs (CVE-2026-20952 and CVE-2026-20953) that can be triggered just by viewing a booby-trapped message in the Preview Pane.

Our October 2025 Patch Tuesday “End of 10” roundup noted that Microsoft had removed a modem driver from all versions after it was discovered that hackers were abusing a vulnerability in it to hack into systems. Adam Barnett at Rapid7 said Microsoft today removed another couple of modem drivers from Windows for a broadly similar reason: Microsoft is aware of functional exploit code for an elevation of privilege vulnerability in a very similar modem driver, tracked as CVE-2023-31096.

“That’s not a typo; this vulnerability was originally published via MITRE over two years ago, along with a credible public writeup by the original researcher,” Barnett said. “Today’s Windows patches remove agrsm64.sys and agrsm.sys. All three modem drivers were originally developed by the same now-defunct third party, and have been included in Windows for decades. These driver removals will pass unnoticed for most people, but you might find active modems still in a few contexts, including some industrial control systems.”

According to Barnett, two questions remain: How many more legacy modem drivers are still present on a fully-patched Windows asset; and how many more elevation-to-SYSTEM vulnerabilities will emerge from them before Microsoft cuts off attackers who have been enjoying “living off the land[line] by exploiting an entire class of dusty old device drivers?”

“Although Microsoft doesn’t claim evidence of exploitation for CVE-2023-31096, the relevant 2023 write-up and the 2025 removal of the other Agere modem driver have provided two strong signals for anyone looking for Windows exploits in the meantime,” Barnett said. “In case you were wondering, there is no need to have a modem connected; the mere presence of the driver is enough to render an asset vulnerable.”

Immersive, Ivanti and Rapid7 all called attention to CVE-2026-21265, which is a critical Security Feature Bypass vulnerability affecting Windows Secure Boot. This security feature is designed to protect against threats like rootkits and bootkits, and it relies on a set of certificates that are set to expire in June 2026 and October 2026. Once these 2011 certificates expire, Windows devices that do not have the new 2023 certificates can no longer receive Secure Boot security fixes.

Barnett cautioned that when updating the bootloader and BIOS, it is essential to prepare fully ahead of time for the specific OS and BIOS combination you’re working with, since incorrect remediation steps can lead to an unbootable system.

“Fifteen years is a very long time indeed in information security, but the clock is running out on the Microsoft root certificates which have been signing essentially everything in the Secure Boot ecosystem since the days of Stuxnet,” Barnett said. “Microsoft issued replacement certificates back in 2023, alongside CVE-2023-24932 which covered relevant Windows patches as well as subsequent steps to remediate the Secure Boot bypass exploited by the BlackLotus bootkit.”

Goettl noted that Mozilla has released updates for Firefox and Firefox ESR resolving a total of 34 vulnerabilities, two of which are suspected to be exploited (CVE-2026-0891 and CVE-2026-0892). Both are resolved in Firefox 147 (MFSA2026-01) and CVE-2026-0891 is resolved in Firefox ESR 140.7 (MFSA2026-03).

“Expect Google Chrome and Microsoft Edge updates this week in addition to a high severity vulnerability in Chrome WebView that was resolved in the January 6 Chrome update (CVE-2026-0628),” Goettl said.

As ever, the SANS Internet Storm Center has a per-patch breakdown by severity and urgency. Windows admins should keep an eye on askwoody.com for any news about patches that don’t quite play nice with everything. If you experience any issues related installing January’s patches, please drop a line in the comments below.

Top

Who Benefited from the Aisuru and Kimwolf Botnets?

Post by Brian Krebs via Krebs on Security »

Our first story of 2026 revealed how a destructive new botnet called Kimwolf has infected more than two million devices by mass-compromising a vast number of unofficial Android TV streaming boxes. Today, we’ll dig through digital clues left behind by the hackers, network operators and services that appear to have benefitted from Kimwolf’s spread.

On Dec. 17, 2025, the Chinese security firm XLab published a deep dive on Kimwolf, which forces infected devices to participate in distributed denial-of-service (DDoS) attacks and to relay abusive and malicious Internet traffic for so-called “residential proxy” services.

The software that turns one’s device into a residential proxy is often quietly bundled with mobile apps and games. Kimwolf specifically targeted residential proxy software that is factory installed on more than a thousand different models of unsanctioned Android TV streaming devices. Very quickly, the residential proxy’s Internet address starts funneling traffic that is linked to ad fraud, account takeover attempts and mass content scraping.

The XLab report explained its researchers found “definitive evidence” that the same cybercriminal actors and infrastructure were used to deploy both Kimwolf and the Aisuru botnet — an earlier version of Kimwolf that also enslaved devices for use in DDoS attacks and proxy services.

XLab said it suspected since October that Kimwolf and Aisuru had the same author(s) and operators, based in part on shared code changes over time. But it said those suspicions were confirmed on December 8 when it witnessed both botnet strains being distributed by the same Internet address at 93.95.112[.]59.

Image: XLab.

RESI RACK

Public records show the Internet address range flagged by XLab is assigned to Lehi, Utah-based Resi Rack LLC. Resi Rack’s website bills the company as a “Premium Game Server Hosting Provider.” Meanwhile, Resi Rack’s ads on the Internet moneymaking forum BlackHatWorld refer to it as a “Premium Residential Proxy Hosting and Proxy Software Solutions Company.”

Resi Rack co-founder Cassidy Hales told KrebsOnSecurity his company received a notification on December 10 about Kimwolf using their network “that detailed what was being done by one of our customers leasing our servers.”

“When we received this email we took care of this issue immediately,” Hales wrote in response to an email requesting comment. “This is something we are very disappointed is now associated with our name and this was not the intention of our company whatsoever.”

The Resi Rack Internet address cited by XLab on December 8 came onto KrebsOnSecurity’s radar more than two weeks before that. Benjamin Brundage is founder of Synthient, a startup that tracks proxy services. In late October 2025, Brundage shared that the people selling various proxy services which benefitted from the Aisuru and Kimwolf botnets were doing so at a new Discord server called resi[.]to.

On November 24, 2025, a member of the resi-dot-to Discord channel shares an IP address responsible for proxying traffic over Android TV streaming boxes infected by the Kimwolf botnet.

When KrebsOnSecurity joined the resi[.]to Discord channel in late October as a silent lurker, the server had fewer than 150 members, including “Shox” — the nickname used by Resi Rack’s co-founder Mr. Hales — and his business partner “Linus,” who did not respond to requests for comment.

Other members of the resi[.]to Discord channel would periodically post new IP addresses that were responsible for proxying traffic over the Kimwolf botnet. As the screenshot from resi[.]to above shows, that Resi Rack Internet address flagged by XLab was used by Kimwolf to direct proxy traffic as far back as November 24, if not earlier. All told, Synthient said it tracked at least seven static Resi Rack IP addresses connected to Kimwolf proxy infrastructure between October and December 2025.

Neither of Resi Rack’s co-owners responded to follow-up questions. Both have been active in selling proxy services via Discord for nearly two years. According to a review of Discord messages indexed by the cyber intelligence firm Flashpoint, Shox and Linus spent much of 2024 selling static “ISP proxies” by routing various Internet address blocks at major U.S. Internet service providers.

In February 2025, AT&T announced that effective July 31, 2025, it would no longer originate routes for network blocks that are not owned and managed by AT&T (other major ISPs have since made similar moves). Less than a month later, Shox and Linus told customers they would soon cease offering static ISP proxies as a result of these policy changes.

Shox and Linux, talking about their decision to stop selling ISP proxies.

DORT & SNOW

The stated owner of the resi[.]to Discord server went by the abbreviated username “D.” That initial appears to be short for the hacker handle “Dort,” a name that was invoked frequently throughout these Discord chats.

Dort’s profile on resi dot to.

This “Dort” nickname came up in KrebsOnSecurity’s recent conversations with “Forky,” a Brazilian man who acknowledged being involved in the marketing of the Aisuru botnet at its inception in late 2024. But Forky vehemently denied having anything to do with a series of massive and record-smashing DDoS attacks in the latter half of 2025 that were blamed on Aisuru, saying the botnet by that point had been taken over by rivals.

Forky asserts that Dort is a resident of Canada and one of at least two individuals currently in control of the Aisuru/Kimwolf botnet. The other individual Forky named as an Aisuru/Kimwolf botmaster goes by the nickname “Snow.”

On January 2 — just hours after our story on Kimwolf was published — the historical chat records on resi[.]to were erased without warning and replaced by a profanity-laced message for Synthient’s founder. Minutes after that, the entire server disappeared.

Later that same day, several of the more active members of the now-defunct resi[.]to Discord server moved to a Telegram channel where they posted Brundage’s personal information, and generally complained about being unable to find reliable “bulletproof” hosting for their botnet.

Hilariously, a user by the name “Richard Remington” briefly appeared in the group’s Telegram server to post a crude “Happy New Year” sketch that claims Dort and Snow are now in control of 3.5 million devices infected by Aisuru and/or Kimwolf. Richard Remington’s Telegram account has since been deleted, but it previously stated its owner operates a website that caters to DDoS-for-hire or “stresser” services seeking to test their firepower.

BYTECONNECT, PLAINPROXIES, AND 3XK TECH

Reports from both Synthient and XLab found that Kimwolf was used to deploy programs that turned infected systems into Internet traffic relays for multiple residential proxy services. Among those was a component that installed a software development kit (SDK) called ByteConnect, which is distributed by a provider known as Plainproxies.

ByteConnect says it specializes in “monetizing apps ethically and free,” while Plainproxies advertises the ability to provide content scraping companies with “unlimited” proxy pools. However, Synthient said that upon connecting to ByteConnect’s SDK they instead observed a mass influx of credential-stuffing attacks targeting email servers and popular online websites.

A search on LinkedIn finds the CEO of Plainproxies is Friedrich Kraft, whose resume says he is co-founder of ByteConnect Ltd. Public Internet routing records show Mr. Kraft also operates a hosting firm in Germany called 3XK Tech GmbH. Mr. Kraft did not respond to repeated requests for an interview.

In July 2025, Cloudflare reported that 3XK Tech (a.k.a. Drei-K-Tech) had become the Internet’s largest source of application-layer DDoS attacks. In November 2025, the security firm GreyNoise Intelligence found that Internet addresses on 3XK Tech were responsible for roughly three-quarters of the Internet scanning being done at the time for a newly discovered and critical vulnerability in security products made by Palo Alto Networks.

Source: Cloudflare’s Q2 2025 DDoS threat report.

LinkedIn has a profile for another Plainproxies employee, Julia Levi, who is listed as co-founder of ByteConnect. Ms. Levi did not respond to requests for comment. Her resume says she previously worked for two major proxy providers: Netnut Proxy Network, and Bright Data.

Synthient likewise said Plainproxies ignored their outreach, noting that the Byteconnect SDK continues to remain active on devices compromised by Kimwolf.

A post from the LinkedIn page of Plainproxies Chief Revenue Officer Julia Levi, explaining how the residential proxy business works.

MASKIFY

Synthient’s January 2 report said another proxy provider heavily involved in the sale of Kimwolf proxies was Maskify, which currently advertises on multiple cybercrime forums that it has more than six million residential Internet addresses for rent.

Maskify prices its service at a rate of 30 cents per gigabyte of data relayed through their proxies. According to Synthient, that price range is insanely low and is far cheaper than any other proxy provider in business today.

“Synthient’s Research Team received screenshots from other proxy providers showing key Kimwolf actors attempting to offload proxy bandwidth in exchange for upfront cash,” the Synthient report noted. “This approach likely helped fuel early development, with associated members spending earnings on infrastructure and outsourced development tasks. Please note that resellers know precisely what they are selling; proxies at these prices are not ethically sourced.”

Maskify did not respond to requests for comment.

The Maskify website. Image: Synthient.

BOTMASTERS LASH OUT

Hours after our first Kimwolf story was published last week, the resi[.]to Discord server vanished, Synthient’s website was hit with a DDoS attack, and the Kimwolf botmasters took to doxing Brundage via their botnet.

The harassing messages appeared as text records uploaded to the Ethereum Name Service (ENS), a distributed system for supporting smart contracts deployed on the Ethereum blockchain. As documented by XLab, in mid-December the Kimwolf operators upgraded their infrastructure and began using ENS to better withstand the near-constant takedown efforts targeting the botnet’s control servers.

An ENS record used by the Kimwolf operators taunts security firms trying to take down the botnet’s control servers. Image: XLab.

By telling infected systems to seek out the Kimwolf control servers via ENS, even if the servers that the botmasters use to control the botnet are taken down the attacker only needs to update the ENS text record to reflect the new Internet address of the control server, and the infected devices will immediately know where to look for further instructions.

“This channel itself relies on the decentralized nature of blockchain, unregulated by Ethereum or other blockchain operators, and cannot be blocked,” XLab wrote.

The text records included in Kimwolf’s ENS instructions can also feature short messages, such as those that carried Brundage’s personal information. Other ENS text records associated with Kimwolf offered some sage advice: “If flagged, we encourage the TV box to be destroyed.”

An ENS record tied to the Kimwolf botnet advises, “If flagged, we encourage the TV box to be destroyed.”

Both Synthient and XLabs say Kimwolf targets a vast number of Android TV streaming box models, all of which have zero security protections, and many of which ship with proxy malware built in. Generally speaking, if you can send a data packet to one of these devices you can also seize administrative control over it.

If you own a TV box that matches one of these model names and/or numbers, please just rip it out of your network. If you encounter one of these devices on the network of a family member or friend, send them a link to this story (or to our January 2 story on Kimwolf) and explain that it’s not worth the potential hassle and harm created by keeping them plugged in.

Top

Valuable News – 2026/01/12

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSDKit is Swift Framework That Provides Overlay for FreeBSD System APIs.
https://github.com/SwiftBSD/FreeBSDKit

FreeBSD – Home NAS – Part 7 – NFSv4 and Linux Clients.
https://rtfm.co.ua/en/freebsd-home-nas-part-7-nfsv4-and-use-with-linux-clients-2/

FreeBSD – Home NAS – Part 8 – NFS and Samba Data Backups Using Restic.
https://rtfm.co.ua/en/freebsd-home-nas-part-8-nfs-and-samba-data-backups-using-restic/

Linuxulator on FreeBSD Feels Like Magic.
https://hayzam.com/blog/02-linuxulator-is-awesome/

pkg: No SRV record found for the repo 'local'.
https://dan.langille.org/2023/07/11/pkg-no-srv-record-found-for-the-repo-local/

FreeBSD Git Weekly: 2025-12-29 to 2026-01-04.
https://freebsd-git-weekly.tarsnap.net/2025-12-29.html

Apache PHP MySQL NetBSD.
https://substack.com/inbox/post/182281112

Using iperf3 and Prometheus for WAN Link Monitoring on FreeBSD.
https://freebsd.uw.cz/2026/01/using-iperf3-and-prometheus-for-wan.html

Using nvi(1) as Minimal and Fast Text Editor.
https://git.sr.ht/~r1w1s1/code-notes/blob/main/notes/NVI_Editor_Guide.txt

You Would Not Believe How Easy It is to Install MediaWiki on FreeBSD.
https://youtube.com/watch?v=WFuNtCw-h5c

Quickshell Adds Initial Support for FreeBSD.
https://github.com/quickshell-mirror/quickshell/commit/6742148cf4a8415a9c51fdeb11d8c3ea716c2e14

Another Classic UNIX is Dead: HP-UX is No Longer Supported.
https://youtube.com/watch?v=ccomS0EzfGk

Idiots at GNOME and Firefox Consider Disabling X11 Middle Click Paste by Default.
https://phoronix.com/news/GNOME-Firefox-MiddleClick-Paste

Ghostty is Now in the FreeBSD Ports Tree/Packages.
https://freshports.org/x11/ghostty/

FreeBSD and Poudriere in High Security Environments.
https://vermaden.wordpress.com/2026/01/07/freebsd-and-poudriere-in-high-security-environments/

GNOME Says Linux Should Be Like Windows and Moves to Disable Middle Click Paste.
https://youtube.com/watch?v=2pnH6FKQxxA

Run FreeBSD/OpenBSD/NetBSD VMs in Incus.
https://gyptazy.com/blog/run-bsd-vms-in-incus/

Installing and Using DosBox-X on FreeBSD.
https://youtube.com/watch?v=5qkE06sv6ZE

Understanding How to Get FreeBSD from Scratch.
https://youtube.com/watch?v=Uc8Z-XX1TS4

Tour of Mandatory Access Control (portacl/biba/mls) on FreeBSD.
https://youtube.com/watch?v=wk7p4bS0-p0

Tour of Mandatory Access Control (mac_do/mac_veriexec) on FreeBSD.
https://youtube.com/watch?v=0Ysw9Ku5cpI

FreeBSD vs. Slackware: Which Super Stable OS is Right for You?
https://zdnet.com/article/freebsd-vs-slackware/

ZFS Management GUI and Web UI ZfDash for Linux/macOS/FreeBSD.
https://github.com/ad4mts/zfdash

Running OpenBSD 7.8 on Your Laptop is Really Hard (Not).
https://k58.uk/openbsd.html

HardenedBSD hbsd-update(8) Signing Material Rotation.
https://groups.google.com/a/hardenedbsd.org/g/users/c/l-UivSvXK5c/m/6hX7SknqBAAJ

BSD Now 645: Unwrapping Gifts/
https://www.bsdnow.tv/645

MidnightBSD 4.0 – Punishment Befitting That of Tantalus.
https://youtube.com/watch?v=tB_x2MiFJA8

Lightweight Dovecot Brute Force doveguard Blocker.
https://github.com/sheridans/doveguard

Add Port to FreeBSD Ports.
https://vermaden.wordpress.com/2026/01/10/add-port-to-freebsd-ports/

Is Not C a Wonderful Language?
http://pigeonsnest.co.uk/stuff/pigeons-device.html

GNUstep Official 2026/01 Presentation.
https://youtube.com/watch?v=DDwLzy8map8

Submitting GitHub Pull Requests to FreeBSD.
https://freebsdfoundation.org/submitting-github-pull-requests-to-freebsd/

BSD or Bust.
https://technophobeconfessions.wordpress.com/2026/01/10/bsd-or-bust/

OMG! I Can Not Believe FreeBSD 15 is Faster than Linux Debian 13.
https://youtube.com/watch?v=BZk6LTfqW30

FreeBSD and Persistent Network Interface Names in VMware.
https://freebsd.uw.cz/2026/01/freebsd-and-persistent-network.html

Quick Look at wc(1) Command in GhostBSD/FreeBSD.
https://youtube.com/watch?v=DQCzfPaQdNc

CheriBSD: FreeBSD for CHERI Enabled Platforms.
https://osnews.com/story/144123/cheribsd-freebsd-for-cheri-enabled-platforms/

OpenBSD on Thinkpad X1 Carbon 9th Gen.
https://xosc.org/x1carbon.html

Performance Fresser Series – Docker vs FreeBSD Jails.
https://linkedin.com/posts/vvoss_performancefresser-docker-freebsd-activity-7416026089275367424-eEk9

Dlaczego Warto Zainstalowac FreeBSD 15 w Home Lab na Proxmox? [Polish]
https://blog.askomputer.pl/instalacja-freebsd-w-proxmox/

Wlasny Serwer FreeBSD – Czesc 3 – PHP i SQL. [Polish]
https://linuxiarze.pl/wlasny-serwer-freebsd-cz-3-php-i-sql/

Hardware

Best ThinkPad (X210Ai) Ever Made is Now Here.
https://youtube.com/watch?v=j6AXBiEdKtI

Bose Open Sources Its Old Smart Speakers Instead of Bricking Them.
https://theverge.com/news/858501/bose-soundtouch-smart-speakers-open-source

Xiaomi XRING O1: Xiaomi Breakthrough in 3nm Chip Technology.
https://pbxscience.com/xiaomi-xring-o1-xiaomi-breakthrough-in-3nm-chip-technology/

Kioxia BG7 Launched for Low Cost M.2 NVMe SSDs.
https://servethehome.com/kioxia-bg7-launched-for-low-cost-m-2-nvme-ssds/

Life

Why Some Empaths Quietly Cut Everyone Off … and Never Return – Carl Jung.
https://youtube.com/watch?v=dYrhhwa1Bz4

Tony Iommi – Godfather of Heavy Metal – Volume One.
https://youtube.com/watch?v=NHZGBfrb6Jw

When World Said No One King Saved 740 Polish Children in 1942.
https://youtube.com/watch?v=GUY5D-1zmLA

Man Charged for Wiping Phone Before CBP Could Search It.
https://404media.co/man-charged-for-wiping-phone-before-cbp-could-search-it/

Food Becoming More Calorific But Less Nutritious.
https://theguardian.com/environment/2025/dec/19/higher-carbon-dioxide-food-more-calorific-less-nutritious-study

Art Of Solitude and Gifts of Aloneness.
https://sacap.edu.za/blog/applied-psychology/overcoming-loneliness/

Other

Code Audit for Tor Project Completed by 7aSecurity.
https://blog.torproject.org/code-audit-network-health-tools/

AMIGA Pointer Archive.
https://heckmeck.de/pointers/

Muffon Open Source Replacement for Spotify.
https://muffon.netlify.app/

Italy Fines Cloudflare €14 Million for Refusing to Filter Pirate Sites on Public 1.1.1.1 DNS.
https://torrentfreak.com/italy-fines-cloudflare-e14-million-for-refusing-to-filter-pirate-sites-on-public-1-1-1-1-dns/

How We Made LEGO SMART Play System.
https://lego.com/en-dk/smart-play/article/innovation

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Add Port to FreeBSD Ports

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

Today I will share a simple way to add software to the FreeBSD Ports tree.

In not so distant past I used shar(1) tool that was part of the FreeBSD Base System to generate information needed to either create new port or to update the old one. It was also one of the officially supported ways to do that … but shar(1) tool is no more – it was removed from the FreeBSD Base System and (fortunately) kept as sysutils/freebsd-shar port.

Now sending the diff from git(1) is needed – but as I read the FreeBSD Porters Handbook – Quick Porting chapter … and tried to follow the instructions one by one … I was not able to generate the needed information. Its not that this official guide is written ‘bad’ on purpose – its probably the ‘missing’ steps are so obvious for FreeBSD maintainers/porters/developers that they even forget to mention them at all.

After some time with the problem I found a simple way that works for me – and today I am sharing that as some of you may also find that information useful.

The Table of Contents is below.

  • Add Port to FreeBSD Ports
  • Create New FreeBSD Port
  • Testing Port Before Submitting
    • Common Problems
      • Checksum Mismatch
      • Size Mismatch
  • Submit Proposal to Add New Port
  • Updating the FreeBSD Port
  • Summary

One may ask WHY I did not submitted the needed changes to the upstream documentation … the reason is disappointment from the past when I tried to fix/update the entire Virtualization chapter of FreeBSD Handbook. 20 years ago I rewrote the chapter and posted it on my (then) place available online. Surprisingly its still available online – FreeBSD Handbook Virtualization – here. I then shared that with people that could update the FreeBSD Handbook with info from there.

To be honest – I did not expected that it would be commited ‘just like that’ as it was – but I at least expected SOME parts of it would be added – as for example the KQEMU part that was missing in the FreeBSD Handbook and as it made QEMU a lot faster it was very valuable … but no. Nothing was added. Nothing was taken from it … like I wasted entire effort … so I keep that FreeBSD Handbook proposal to remind me about it. After I finally succeeded with my 3rd blog attempt I do not care anymore for such things – I just ‘dump’ all the info I have on the blog and if someone wants to add it to the official documentation – like with Jails chapter of FreeBSD Handbook – great – but I am done with official proposals to omit disappointment.

Now to the point.

Create New FreeBSD Port

I will create an example based on my sysutils/lsblk port.

when I proposed to add lsblk(8) as 1.0 version the Makefile and distinfo and pkg-descr files looked like that:

FreeBSD % cat Makefile
PORTNAME=	lsblk
PORTVERSION=	1.0
CATEGORIES=	sysutils
MASTER_SITES=	https://github.com/vermaden/lsblk/raw/master/release/
MAINTAINER=	vermaden@interia.pl
COMMENT=	Lists information about block devices in the system
LICENSE=	BSD2CLAUSE
NO_BUILD=	YES
NO_ARCH=	YES
PLIST_FILES=	sbin/${PORTNAME}
do-install:
	${INSTALL_SCRIPT} ${WRKSRC}/lsblk.sh \
		${STAGEDIR}${PREFIX}/sbin/${PORTNAME}
.include 

FreeBSD % cat distinfo
TIMESTAMP = 1570494909
SHA256 (lsblk-1.0.tar.gz) = ec6335ec27fd7ec1c56b2700d073c659c1704b816aa08c0f4d831d417dd2affb
SIZE (lsblk-1.0.tar.gz) = 4338

FreeBSD % cat pkg-descr
Lists information about block devices in the system in a similar way
that util-linux's lsblk(8) does.
WWW: https://github.com/vermaden/scripts/blob/master/lsblk.sh

You can even view them in the FreeBSD Ports history here:

To generate the distinfo file you may use my ports-generate-distinfo.sh script.

To make submit a port you will also need lsblk-1.0.tar.gz file placed in the https://github.com/vermaden/lsblk/release/ path – and it contains these contents as shown below.

Just a lsblk-1.0 dir and lsblk.sh in it.

Testing Port Before Submitting

Before you send the ‘BUG’ report to create a port first test the port locally.

FreeBSD # tree /usr/ports/sysutils/lsblk
/usr/ports/sysutils/lsblk
├── distinfo
├── Makefile
└── pkg-descr

1 directory, 3 files

FreeBSD # cd /usr/ports/sysutils/lsblk

FreeBSD # pwd
/usr/ports/sysutils/lsblk

FreeBSD # make clean distclean
===>  Cleaning for lsblk-1.0
===>  Deleting distfiles for lsblk-1.0

FreeBSD # make
===>   lsblk-1.0 depends on file: /usr/local/sbin/pkg - found
=> lsblk-1.0.tar.gz doesn't seem to exist in /usr/ports/distfiles.
=> Attempting to fetch https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz
lsblk-1.0.tar.gz                                      4338  B   13 MBps    00s
===> Fetching all distfiles required by lsblk-1.0 for building
===>  Extracting for lsblk-1.0
=> SHA256 Checksum OK for lsblk-1.0.tar.gz.
===>  Patching for lsblk-1.0
===>  Configuring for lsblk-1.0
===>  Staging for lsblk-1.0
===>   Generating temporary packing list
install  -m 555 /usr/ports/obj/usr/ports/sysutils/lsblk/work/lsblk-1.0/lsblk.sh  /usr/ports/obj/usr/ports/sysutils/lsblk/work/stage/usr/local/sbin/lsblk
====> Compressing man pages (compress-man)

Common Problems

Below are common problems that you may encounter while preparing the port.

Checksum Mismatch

More often in the updating then creating first one – but still.

FreeBSD # make
===>   lsblk-1.0 depends on file: /usr/local/sbin/pkg - found
=> lsblk-1.0.tar.gz doesn't seem to exist in /usr/ports/distfiles.
=> Attempting to fetch https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz
lsblk-1.0.tar.gz                                      4338  B   26 MBps    00s
===> Fetching all distfiles required by lsblk-1.0 for building
===>  Extracting for lsblk-1.0
=> SHA256 Checksum mismatch for lsblk-1.0.tar.gz.
===>  Refetch for 1 more times files:  lsblk-1.0.tar.gz 
===>   lsblk-1.0 depends on file: /usr/local/sbin/pkg - found
=> lsblk-1.0.tar.gz doesn't seem to exist in /usr/ports/distfiles.
=> Attempting to fetch https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz
lsblk-1.0.tar.gz                                      4338  B   12 MBps    00s
===> Fetching all distfiles required by lsblk-1.0 for building
===>   lsblk-1.0 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by lsblk-1.0 for building
=> SHA256 Checksum mismatch for lsblk-1.0.tar.gz.
===>  Giving up on fetching files:  lsblk-1.0.tar.gz 
Make sure the Makefile and distinfo file (/usr/ports/sysutils/lsblk/distinfo)
are up to date.  If you are absolutely sure you want to override this
check, type "make NO_CHECKSUM=yes [other args]".
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/sysutils/lsblk
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/lsblk


Check how to fix that.

FreeBSD # sha256 /usr/ports/distfiles/lsblk-1.0.tar.gz
SHA256 (/usr/ports/distfiles/lsblk-1.0.tar.gz) = ec6335ec27fd7ec1c56b2700d073c659c1704b816aa08c0f4d831d417dd2affb

FreeBSD # grep SHA /usr/ports/sysutils/lsblk/distinfo
SHA256 (lsblk-1.0.tar.gz) = ec6335ec27fd7ec1c56b2700d073c659c1704b816aa08c0f4d831d417dd2affba

As You can see there is additional ‘a’ at the end that makes the checksum different.

Size Mismatch

Also typical error.

FreeBSD # make
===>   lsblk-1.0 depends on file: /usr/local/sbin/pkg - found
=> lsblk-1.0.tar.gz doesn't seem to exist in /usr/ports/distfiles.
=> Attempting to fetch https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz
fetch: https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz: size mismatch: expected 4333, actual 4338
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/lsblk-1.0.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/lsblk-1.0.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles and try again.
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/lsblk

Lets check the size.

FreeBSD # fetch https://github.com/vermaden/lsblk/raw/master/release/lsblk-1.0.tar.gz
lsblk-1.0.tar.gz                                      4338  B   27 MBps    00s

FreeBSD # stat -f "%z" lsblk-1.0.tar.gz
4338

FreeBSD # grep SIZE /usr/ports/sysutils/lsblk/distinfo
SIZE (lsblk-1.0.tar.gz) = 4333

Yep – wrong size.

Submit Proposal to Add New Port

Now – to add proposal to add a port to FreeBSD Ports you will need to generate git(1) diff for that.

Below are steps needed to get one.

First we need to get the FreeBSD Ports tree.

FreeBSD % git clone https://git.freebsd.org/ports.git freebsd-ports
Cloning into 'freebsd-ports'...
remote: Enumerating objects: 6635778, done.
remote: Counting objects: 100% (995/995), done.
remote: Compressing objects: 100% (177/177), done.
remote: Total 6635778 (delta 925), reused 816 (delta 816), pack-reused 6634783 (from 1)
Receiving objects: 100% (6635778/6635778), 1.45 GiB | 1.89 MiB/s, done.
Resolving deltas: 100% (4017143/4017143), done.
Updating files: 100% (166471/166471), done.

FreeBSD % cd freebsd-ports

Now you need to create new branch – name it as You like it – I will use sysutils-lsblk one without better idea.

FreeBSD % git checkout -b sysutils-lsblk
Switched to a new branch 'sysutils-lsblk'

Now copy your content into that local tree.

FreeBSD % cp -avp ~/misc/git/lsblk/port/sysutils-lsblk sysutils/lsblk
/home/vermaden/misc/git/lsblk/port/sysutils-lsblk -> sysutils/lsblk
/home/vermaden/misc/git/lsblk/port/sysutils-lsblk/distinfo -> sysutils/lsblk/distinfo
/home/vermaden/misc/git/lsblk/port/sysutils-lsblk/Makefile -> sysutils/lsblk/Makefile
/home/vermaden/misc/git/lsblk/port/sysutils-lsblk/pkg-descr -> sysutils/lsblk/pkg-descr

Now add it with git(1) command and create that diff.

FreeBSD % git add sysutils/lsblk

FreeBSD % git diff --staged > /home/vermaden/sysutils-lsblk.diff

This is how our git(1) diff looks like.

FreeBSD % cat ~/sysutils-lsblk.diff
diff --git a/sysutils/lsblk/Makefile b/sysutils/lsblk/Makefile
new file mode 100644
index 000000000000..f59e4af7948b
--- /dev/null
+++ b/sysutils/lsblk/Makefile
@@ -0,0 +1,15 @@
+PORTNAME=	lsblk
+PORTVERSION=	1.0
+CATEGORIES=	sysutils
+MASTER_SITES=	https://github.com/vermaden/scripts/raw/master/release/
+MAINTAINER=	vermaden@interia.pl
+COMMENT=	Lists information about block devices in the system
+LICENSE=	BSD2CLAUSE
+NO_BUILD=	YES
+NO_ARCH=	YES
+PLIST_FILES=	sbin/${PORTNAME}
+do-install:
+	${INSTALL_SCRIPT} ${WRKSRC}/lsblk.sh \
+		${STAGEDIR}${PREFIX}/sbin/${PORTNAME}
+.include 
+
diff --git a/sysutils/lsblk/distinfo b/sysutils/lsblk/distinfo
new file mode 100644
index 000000000000..bacd71658c5a
--- /dev/null
+++ b/sysutils/lsblk/distinfo
@@ -0,0 +1,4 @@
+TIMESTAMP = 1570494909
+SHA256 (lsblk-1.0.tar.gz) = ec6335ec27fd7ec1c56b2700d073c659c1704b816aa08c0f4d831d417dd2affb
+SIZE (lsblk-1.0.tar.gz) = 4338
+
diff --git a/sysutils/lsblk/pkg-descr b/sysutils/lsblk/pkg-descr
new file mode 100644
index 000000000000..01e1707486ed
--- /dev/null
+++ b/sysutils/lsblk/pkg-descr
@@ -0,0 +1,4 @@
+Lists information about block devices in the system in a similar way
+that util-linux's lsblk(8) does.
+WWW: https://github.com/vermaden/scripts/blob/master/lsblk.sh
+

… and this sysutils-lsblk.diff we would have to attach to our ‘BUG’ report for new FreeBSD port.

Now go to the https://freebsd.org/send-pr.html page (you will be redirected to https://bugs.freebsd.org/bugzilla/enter_bug.cgi page) and click Ports & Packages type … and fill all the needed information. On the screenshot you have everything that is needed.

Remember to add the sysutils-lsblk.diff with Add an Attachment button below.

If some modifications would be needed – and often are – FreeBSD developers and maintainers will tell you what to do 🙂

Updating the FreeBSD Port

Now … after You already have a FreeBSD port there will be a need to update it.

The procedure is generally the same – you fetch the FreeBSD Ports tree – create new branch – dump your updates to it – create git(1) diff and attach it to the ‘BUG’ report.

Example report could look like that one below.

Summary

Some notes of the current Status Quo and trends.

For some reason instead of just links to where to fetch the code from the FreeBSD implemented additional variables just for GitHub to … make it more simple? Not for me at least … and its just for one vendor. IMHO its pointless to create such ‘per vendor’ settings while we can have just generic things that works for everyone … but that is me.

Example Makefile for sysutils/jmore with these options for GitHub is below.

FreeBSD % cat /usr/ports/*/jmore/Makefile
PORTNAME=       jmore
DISTVERSION=    0.5.20251030
DISTVERSIONSUFFIX= -g${GH_TAGNAME}
CATEGORIES=     sysutils

MAINTAINER=     vermaden@interia.pl
COMMENT=        FreeBSD Jail listing and managing utility
WWW=            https://github.com/vermaden/jmore

LICENSE=        BSD2CLAUSE
LICENSE_FILE=   ${WRKSRC}/LICENSE

USE_GITHUB=     yes
GH_ACCOUNT=     vermaden
GH_PROJECT=     jmore
GH_TAGNAME=     9b72e97

NO_ARCH=        YES
NO_BUILD=       YES

PLIST_FILES=    sbin/${PORTNAME}

do-install:
        ${INSTALL_SCRIPT} ${WRKSRC}/jmore \
        ${STAGEDIR}${PREFIX}/sbin/${PORTNAME}

.include 

Now nothing is stopping You from adding something to the FreeBSD Ports tree.

EOF
Top

FreeBSD and Poudriere in High Security Environments

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

Most of the time FreeBSD systems are used with wide open connection to the Internet along with fully working DNS that resolves anything the Root Servers resolve … but FreeBSD – besides being used as SONY PlayStation gaming systems or Netflix storage layer.

Its also used in high security environments without any external DNS access or direct Internet connection to the outside World … yet the security patches are fetched and applied and custom PKGBASE and/or Poudriere systems build base system/packages while fetching them from the Internet over some dedicated proxy.

Many people will not read entire article so I will point that in the beginning – that I am really grateful to Mariusz Zaborski (oshogbo) for his help with this one – without his help – it just would not happen.

By default FreeBSD does not work well in such environments … in this article we will configure FreeBSD to make everything work as needed.

The Table of Contents is as follows.

  • FreeBSD and Poudriere in High Security Environments
  • Example Proxy Configuration
  • Physical (or Virtual) FreeBSD Host
    • pkg(8)
    • FreeBSD Ports Tree
    • Proxy on the Fly
    • Back to the PKGBASE
  • Poudriere in Proxy World
    • Basic Poudriere Setup
    • Important Poudriere Config Part

Example Proxy Configuration

For completeness I will add Squid configuration used here – so that all information will be available.

proxy # grep '^[^#]' /usr/local/etc/squid/squid.conf
http_port 127.0.0.1:3128
http_port 10.0.0.41:3128
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
acl localnet src 127.0.0.1
acl localnet src 10.0.0.0/8     # RFC 1918 local private network (LAN)
acl localnet src 172.16.0.0/12  # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
http_access allow localnet
http_access allow localhost
http_access deny all
visible_hostname proxy.xyz
acl custom-local dstdomain .custom.xyz
cache_peer 10.0.0.42 parent 3128 0 no-query default name=weathertop
cache_peer_domain 10.0.0.43 !.xyz
never_direct deny custom-local
never_direct allow all
cache_dir ufs /var/local/squid/cache 50 16 256
coredump_dir /var/local/squid/cache
access_log stdio:/var/local/log/squid/access.log
cache_store_log stdio:/var/local/log/squid/store.log
cache_log /var/local/log/squid/cache.log
refresh_pattern ^ftp:          1440 20% 10080
refresh_pattern ^gopher:       1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%     0
refresh_pattern .                 0 20%  4320
email_err_data off

Now we will configure a FreeBSD host to use it properly.

Physical (or Virtual) FreeBSD Host

For a start we will make pkg(8) work with our proxy system.

pkg(8)

I installed that FreeBSD 15.0-RELEASE system with PKGBASE way – Brave New PKGBASE World – described in details here. With offline install using PKGBASE packages from the install disc1.iso medium.

That means the pkg(8) is already bootstrapped … we will turn that ‘OFF’ for a moment.

test # pkg info
FreeBSD-acct-15.0              System resource accounting
FreeBSD-acpi-15.0              Advanced Configuration and Power Interface (ACPI) utilities
FreeBSD-apm-15.0               Intel / Microsoft APM BIOS utility
(...)
FreeBSD-zlib-lib32-15.0        DEFLATE (gzip) data compression library (32-bit libraries)
FreeBSD-zoneinfo-15.0          Timezone database
pkg-2.4.2                      Package manager

test # mv /var/db/pkg /var/db/pkg.BCK
test # mv /usr/local  /usr/local.BCK

Now if you would like to bootstrap pkg(8) it will fail.

test # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly, please wait...
pkg: Error fetching https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/Latest/pkg.pkg: Transient resolver failure
A pre-built version of pkg could not be found for your system.
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0, please wait...
pkg: Error fetching https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/Latest/pkg.pkg: Transient resolver failure
A pre-built version of pkg could not be found for your system.

Now we will export(1) needed proxy setting into environment.

test # export HTTP_PROXY="http://10.0.0.41:3128" 

test # export HTTPS_PROXY="http://10.0.0.41:3128"

test # export FTP_PROXY="http://10.0.0.41:3128"

test # env | grep -i proxy
HTTP_PROXY=http://10.0.0.41:3128
HTTPS_PROXY=http://10.0.0.41:3128
FTP_PROXY=http://10.0.0.41:3128

If you want to make it permanent for the default sh(1) shell then do this.

test # cat < /etc/profile.d/proxy.sh
export HTTP_PROXY=http://10.0.0.41:3128
export HTTPS_PROXY=http://10.0.0.41:3128
export FTP_PROXY=http://10.0.0.41:3128
EOF

Now with each new login these proxy settings will be available.

Lets try with pkg(8) again.

test # pkg info
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
Installing pkg-2.4.2...
Extracting pkg-2.4.2: 100%
pkg-2.4.2                      Package manager

The pkg(8) has now bootstrap completed and will work, right? Right?

test # pkg update
Updating FreeBSD-ports repository catalogue...
pkg: No SRV record found for the repo 'FreeBSD-ports'
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository FreeBSD-ports
Updating FreeBSD-ports-kmods repository catalogue...
pkg: No SRV record found for the repo 'FreeBSD-ports-kmods'
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository FreeBSD-ports-kmods
Error updating repositories!

Small modification is needed. One need to remove pkg+ prefix from all url: paths and to switch mirror_type: from srv to none. Then it will work.

Now … the reason why FreeBSD uses the pkg+ prefix is this:

  • pkg+https:// tells pkg(8) to use libpkg internal HTTP/HTTPS fetcher.
  • https:// tells pkg(8) to use external fetcher – usually with FreeBSD fetch(1) tool.

Now … the needed changes.

test # \
  grep '^[^#]' /etc/pkg/FreeBSD.conf \
    | sed -e 's.pkg+..g' \
          -e 's."srv"."none".g' \
          -e 's.enabled: no.enabled: yes.g' \
    > /root/FreeBSD.conf 

The diff(1) for that change is below.

root # diff -u /etc/pkg/FreeBSD.conf /root/FreeBSD.conf 
--- /etc/pkg/FreeBSD.conf  2025-11-28 00:00:00.000000000 +0000
+++ /root/FreeBSD.conf     2026-01-07 00:11:41.534051000 +0000
@@ -10,23 +10,23 @@
 #
 
 FreeBSD-ports: {
-  url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly",
-  mirror_type: "srv",
+  url: "https://pkg.FreeBSD.org/${ABI}/quarterly",
+  mirror_type: "none",
   signature_type: "fingerprints",
   fingerprints: "/usr/share/keys/pkg",
   enabled: yes
 }
 FreeBSD-ports-kmods: {
-  url: "pkg+https://pkg.FreeBSD.org/${ABI}/kmods_quarterly_${VERSION_MINOR}",
-  mirror_type: "srv",
+  url: "https://pkg.FreeBSD.org/${ABI}/kmods_quarterly_${VERSION_MINOR}",
+  mirror_type: "none",
   signature_type: "fingerprints",
   fingerprints: "/usr/share/keys/pkg",
   enabled: yes
 }
 FreeBSD-base: {
-  url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_release_${VERSION_MINOR}",
-  mirror_type: "srv",
+  url: "https://pkg.FreeBSD.org/${ABI}/base_release_${VERSION_MINOR}",
+  mirror_type: "none",
   signature_type: "fingerprints",
   fingerprints: "/usr/share/keys/pkgbase-${VERSION_MAJOR}",
-  enabled: no
+  enabled: yes
 }

Now – lets leave the original /etc/pkg/FreeBSD.conf unmodified and create /usr/local/etc/pkg/repos/FreeBSD.conf that will override the defaults.

test # mkdir -pv /usr/local/etc/pkg/repos
/usr/local/etc/pkg
/usr/local/etc/pkg/repos

test # cp /root/FreeBSD.conf /usr/local/etc/pkg/repos/

Now that pkg(8) should work well over proxy.

test # pkg update 
Updating FreeBSD-ports repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   10 MiB   3.6MB/s    00:03    
Processing entries: 100%
FreeBSD-ports repository update completed. 36390 packages processed.
Updating FreeBSD-ports-kmods repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   31 KiB  31.3kB/s    00:01    
Processing entries: 100%
FreeBSD-ports-kmods repository update completed. 204 packages processed.
Updating FreeBSD-base repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   80 KiB  81.5kB/s    00:01    
Processing entries: 100%
FreeBSD-base repository update completed. 496 packages processed.

Lets try to actually install any software.

test # pkg install lsblk beadm 
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-ports-kmods repository catalogue...
FreeBSD-ports-kmods repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        beadm: 1.3.5_1 [FreeBSD-ports]
        lsblk: 4.0 [FreeBSD-ports]
        gitup: 1.0 [FreeBSD-ports]

Number of packages to be installed: 3

18 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/3] Fetching lsblk-4.0~3110a4bb46.pkg: 100%    7 KiB   7.2kB/s    00:01    
[2/3] Fetching beadm-1.3.5_1~53f06720d4.pkg: 100%   11 KiB  11.0kB/s    00:01    
[3/3] Fetching gitup-1.0~2c88a1f1f1.pkg: 100%   36 KiB  37.0kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Installing beadm-1.3.5_1...
[1/3] Extracting beadm-1.3.5_1: 100%
[2/3] Installing lsblk-4.0...
[2/3] Extracting lsblk-4.0: 100%
[3/3] Installing gitup-1.0...
[3/3] Extracting gitup-1.0: 100%

test # lsblk -d
DEVICE SIZE MODEL
nda0    10G bhyve-NVMe
-       10G TOTAL SYSTEM STORAGE

Works.

If for some reason the above method will not work – you may also configure proxy server within the pkg(8) config /usr/local/etc/pkg.conf file.

test # tail -5 /usr/local/etc/pkg.conf
PKG_ENV {
  HTTP_PROXY: "http://10.0.0.41:3128"
  HTTPS_PROXY: "https://10.0.0.41:3128"
  FTP_PROXY: "http://10.0.0.41:3128"
}

These settings will also override any ‘system’ settings we set previously in the /etc/profile.d/proxy.sh file.

FreeBSD Ports Tree

We already installed gitup tool that will allow us to update the FreeBSD Ports tree easily.

Its default config is more then enought.

test # grep -m 1 -A 6 ports /usr/local/etc/gitup.conf
        "ports" : {
                "repository_path"  : "/ports.git",
                "branch"           : "main",
                "target_directory" : "/usr/ports",
                "ignores"          : [],
        },

Lets try fetching the FreeBSD Ports tree now.

test # gitup ports
# Host: git.freebsd.org
# Port: 443
# Proxy Host: 10.0.0.41
# Proxy Port: 3128
# Repository Path: /ports.git
# Target Directory: /usr/ports
# Want: 284813ec0382a2bfe5b2e74a3081a67599d3155d
# Branch: main
# Action: clone
  75 MB in 0m25s, 4614 kB/s now 
 + /usr/ports/.arcconfig
 + /usr/ports/.gitignore
 + /usr/ports/.hooks/pre-commit
(...)
 + /usr/ports/x11/zutty/Makefile
 + /usr/ports/x11/zutty/distinfo
 + /usr/ports/x11/zutty/pkg-descr
#
# Please review the following file(s) for important changes.
#       /usr/ports/UPDATING
#       /usr/ports/mail/dspam/files/UPDATING
#
# Done.

Seems to work.

To now update the FreeBSD Ports tree run gitup(1) command again.

test # gitup ports
# Scanning local repository...
# Host: git.freebsd.org
# Port: 443
# Proxy Host: 10.0.0.41
# Proxy Port: 3128
# Repository Path: /ports.git
# Target Directory: /usr/ports
# Have: 284813ec0382a2bfe5b2e74a3081a67599d3155d
# Want: 7b2f3c4f484b1634066997a91836554608c72c48
# Branch: main
# Action: pull
 * /usr/ports/lang/spidermonkey115/Makefile
 * /usr/ports/lang/spidermonkey115/distinfo
 * /usr/ports/lang/spidermonkey140/Makefile
 * /usr/ports/lang/spidermonkey140/distinfo
# Done.

If for some reason You will find gitup(1) or git(1) does not work – you may always configure system wide proxy as follows.

test # git config --system http.proxy http://10.0.0.41:3128

That would help.

Proxy on the Fly

If for some reason You will need to force the proxy settings for a single command – then use something like that below.

test # \
  env HTTP_PROXY="http://10.0.0.41:3128"  \
      HTTPS_PROXY="http://10.0.0.41:3128" \
      FTP_PROXY="http://10.0.0.41:3128" 
    command ...

Back to the PKGBASE

Now – lets bring back our original pkg(8) config – we may ‘keep’ the current ‘test’ bootstrap if needed with .CUSTOM suffix.

test # mv /usr/local  /usr/local.CUSTOM
test # mv /var/db/pkg /var/db/pkg.CUSTOM

test # mv /usr/local.BCK  /usr/local
test # mv /var/db/pkg.BCK /var/db/pkg

Now the ‘original’ pkg(8) config that keeps PKGBASE information works again.

test # pkg info | (head -3 ;echo '(...)'; tail -3)
FreeBSD-acct-15.0              System resource accounting
FreeBSD-acpi-15.0              Advanced Configuration and Power Interface (ACPI) utilities
FreeBSD-apm-15.0               Intel / Microsoft APM BIOS utility
(...)
FreeBSD-zlib-lib32-15.0        DEFLATE (gzip) data compression library (32-bit libraries)
FreeBSD-zoneinfo-15.0          Timezone database
pkg-2.4.2                      Package manager

But when we will now try to update the pkg(8) repositories it will fail … why?

test # pkg update
Updating FreeBSD-ports repository catalogue...
pkg: No SRV record found for the repo 'FreeBSD-ports'
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository FreeBSD-ports
Updating FreeBSD-ports-kmods repository catalogue...
pkg: No SRV record found for the repo 'FreeBSD-ports-kmods'
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository FreeBSD-ports-kmods
Error updating repositories!

Its because our config override was placed in /usr/local path … and we just wiped that away.

Copy working proxy config again then.

test # mkdir -pv /usr/local/etc/pkg/repos
/usr/local/etc/pkg
/usr/local/etc/pkg/repos

test # cp /root/FreeBSD.conf /usr/local/etc/pkg/repos/

Now update will work again.

test # pkg update
Updating FreeBSD-ports repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   10 MiB   5.4MB/s    00:02    
Processing entries: 100%
FreeBSD-ports repository update completed. 36390 packages processed.
Updating FreeBSD-ports-kmods repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   31 KiB  31.3kB/s    00:01    
Processing entries: 100%
FreeBSD-ports-kmods repository update completed. 204 packages processed.
Updating FreeBSD-base repository catalogue...
pkg: Repository FreeBSD-base has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   80 KiB  81.5kB/s    00:01    
Processing entries: 100%
FreeBSD-base repository update completed. 496 packages processed.
All repositories are up to date.

You can even do PKGBASE upgrade if wanted.

test # pkg upgrade
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-ports-kmods repository catalogue...
FreeBSD-ports-kmods repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
Checking for upgrades (4 candidates): 100%
Processing candidates (4 candidates): 100%
The following 4 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        FreeBSD-kernel-generic: 15.0 -> 15.0p1 [FreeBSD-base]
        FreeBSD-rescue: 15.0 -> 15.0p1 [FreeBSD-base]
        FreeBSD-runtime: 15.0 -> 15.0p1 [FreeBSD-base]
        FreeBSD-utilities: 15.0 -> 15.0p1 [FreeBSD-base]

Number of packages to be upgraded: 4

62 MiB to be downloaded.

Proceed with this action? [y/N]: 

Poudriere in Proxy World

Now to another deeper level – like in Inception (2010) movie – the Poudriere package building harvester.

If you want to check more on the Poudriere itself you can check these:

Now – first we need to install poudriere-devel package as it has the latest features.

test # pkg install poudriere-devel ccache4 git nginx
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-ports-kmods repository catalogue...
FreeBSD-ports-kmods repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 28 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        brotli: 1.1.0,1 [FreeBSD-ports]
        ccache4: 4.10.2_1 [FreeBSD-ports]
        curl: 8.16.0 [FreeBSD-ports]
        expat: 2.7.3 [FreeBSD-ports]
        gettext-runtime: 0.23.1 [FreeBSD-ports]
        git: 2.51.0 [FreeBSD-ports]
        indexinfo: 0.3.1_1 [FreeBSD-ports]
        libffi: 3.5.1 [FreeBSD-ports]
        libidn2: 2.3.8 [FreeBSD-ports]
        libnghttp2: 1.67.0 [FreeBSD-ports]
        libpsl: 0.21.5_2 [FreeBSD-ports]
        libssh2: 1.11.1,3 [FreeBSD-ports]
        libunistring: 1.4.1 [FreeBSD-ports]
        mpdecimal: 4.0.1 [FreeBSD-ports]
        nginx: 1.28.0_3,3 [FreeBSD-ports]
        p5-Authen-SASL: 2.1900 [FreeBSD-ports]
        p5-Crypt-URandom: 0.54 [FreeBSD-ports]
        p5-Digest-HMAC: 1.05 [FreeBSD-ports]
        p5-Error: 0.17030 [FreeBSD-ports]
        p5-IO-Socket-SSL: 2.095 [FreeBSD-ports]
        p5-MIME-Base32: 1.303 [FreeBSD-ports]
        p5-MIME-Base64: 3.16 [FreeBSD-ports]
        p5-Mozilla-CA: 20250602 [FreeBSD-ports]
        p5-Net-SSLeay: 1.94 [FreeBSD-ports]
        p5-URI: 5.32_1 [FreeBSD-ports]
        poudriere-devel: 3.4.99.20251213 [FreeBSD-ports]
        python311: 3.11.14 [FreeBSD-ports]
        readline: 8.2.13_2 [FreeBSD-ports]

Number of packages to be installed: 28

The process will require 283 MiB more space.
41 MiB to be downloaded.

Proceed with this action? [y/N]: y
[1/25] Fetching mpdecimal-4.0.1~f774e949d8.pkg: 100%  157 KiB 160.5kB/s    00:01    
(...)
[28/28] Installing git-2.51.0...
===> Creating groups
Creating group 'git_daemon' with gid '964'
===> Creating users
Creating user 'git_daemon' with uid '964'
[28/28] Extracting git-2.51.0: 100%

Basic Poudriere Setup

We will now setup some basic Poudriere setup.

test # export SSL=/usr/local/etc/ssl

test # mkdir -p \
              /usr/ports/distfiles \
              ${SSL}/keys \
              ${SSL}/certs

test # chmod 0600 ${SSL}/keys

test # openssl genrsa -out ${SSL}/keys/poudriere.key 4096

test # openssl rsa \
              -in  ${SSL}/keys/poudriere.key -pubout \
              -out ${SSL}/certs/poudriere.cert

test # mkdir /var/ccache

test # cat < /usr/local/etc/poudriere.conf
ZPOOL=zroot
FREEBSD_HOST=ftp://ftp.freebsd.org
BASEFS=/usr/local/poudriere
POUDRIERE_DATA=/usr/local/poudriere/data
DISTFILES_CACHE=/usr/ports/distfiles
CCACHE_DIR=/var/ccache
CHECK_CHANGED_OPTIONS=verbose
CHECK_CHANGED_DEPS=yes
PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key
URL_BASE=http://0.0.0.0/
USE_TMPFS=no
TMPFS_LIMIT=12
MAX_MEMORY=12
PARALLEL_JOBS=4
PREPARE_PARALLEL_JOBS=4
MAX_FILES=4096
KEEP_OLD_PACKAGES=yes
KEEP_OLD_PACKAGES_COUNT=3
CHECK_CHANGED_OPTIONS=verbose
CHECK_CHANGED_DEPS=yes
RESTRICT_NETWORKING=no
PACKAGE_FETCH_URL="http://pkg.FreeBSD.org/\${ABI}"
PACKAGE_FETCH_BRANCH="latest"
export HTTP_PROXY="http://10.0.0.41:3128"
export HTTPS_PROXY="http://10.0.0.41:3128"
export FTP_PROXY="http://10.0.0.41:3128"
EOF

test # mkdir -p /usr/local/poudriere/data/logs/bulk

test # ln -s \
              /usr/local/etc/ssl/certs/poudriere.cert \
              /usr/local/poudriere/data/logs/bulk/poudriere.cert

test # cat < /usr/local/etc/poudriere.d/make.conf
# general
ALLOW_UNSUPPORTED_SYSTEM=yes
DISABLE_LICENSES=yes

# ccache(1)
WITH_CCACHE_BUILD=yes

# ports options
FORCE_MAKE_JOBS=yes
MAKE_JOBS_UNSAFE=yes
MAKE_JOBS_NUMBER=8
EOF

test # sed -i '' -E 's|text/plain[\t\ ]*txt|text/plain txt log|g' /usr/local/etc/nginx/mime.types

test # cat < /usr/local/etc/nginx/nginx.conf
events {
  worker_connections 1024;
}

http {
  include      mime.types;
  default_type application/octet-stream;

  server {
    listen 80 default;
    server_name 0.0.0.0;
    root /usr/local/share/poudriere/html;

    location /data {
      alias /usr/local/poudriere/data/logs/bulk;
      autoindex on;
    }

    location /packages {
      root /usr/local/poudriere/data;
      autoindex on;
    }
  }
}
EOF

test # mkdir /root/.cache

test # ln -sf /var/ccache /root/.cache/ccache

test # cat < /var/ccache/ccache.conf
max_size = 0
cache_dir = /var/ccache
base_dir = /var/ccache
hash_dir = false
EOF

Important Poudriere Config Part

The IMPORTANT settings here – to allow Poudriere function properly within proxy environment – are these five lines in the /usr/local/etc/poudriere.conf file.

export HTTP_PROXY="http://10.0.0.41:3128"
export HTTPS_PROXY="http://10.0.0.41:3128"
export FTP_PROXY="http://10.0.0.41:3128"
PACKAGE_FETCH_URL="http://pkg.FreeBSD.org/\${ABI}"
PACKAGE_FETCH_BRANCH="latest"

The first three are obvious – and YES – they need the export prefix to work.

The other two are less obvious … and I will show You why in a moment.

We need to create some FreeBSD jail – we will use 14.3-RELEASE as example. Use any version that you will be building packages for.

test # poudriere jail -c -j 14-3-R-amd64 -v 14.3-RELEASE
[00:00:00] Creating 14-3-R-amd64 fs at /usr/local/poudriere/jails/14-3-R-amd64... done
[00:00:00] Using pre-distributed MANIFEST for FreeBSD 14.3-RELEASE amd64
[00:00:00] Fetching base for FreeBSD 14.3-RELEASE amd64
base.txz                                               200 MB 5128 kBps    41s
[00:00:41] Extracting base... done
[00:00:49] Fetching src for FreeBSD 14.3-RELEASE amd64
src.txz                                                206 MB 4828 kBps    44s
[00:01:34] Extracting src... done
[00:01:46] Fetching lib32 for FreeBSD 14.3-RELEASE amd64
lib32.txz                                               60 MB   11 MBps    05s
[00:01:52] Extracting lib32... done
[00:01:54] Cleaning up... done
[00:01:54] Recording filesystem state for clean... done
[00:01:54] Upgrading using http
Looking up update.FreeBSD.org mirrors... none found.
Fetching public key from update.FreeBSD.org... done.
Fetching metadata signature for 14.3-RELEASE from update.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 2 metadata files... done.
Inspecting system... done.
Preparing to download files... done.
Fetching 196 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150....160....170....180....190... done.
Applying patches... done.
Fetching 40 files... ....10....20....30....40 done.
The following files will be removed as part of updating to
14.3-RELEASE-p7:
/usr/src/contrib/libarchive/libarchive/archive_getdate.c
/usr/src/contrib/libarchive/libarchive/archive_getdate.h
/usr/src/contrib/libarchive/libarchive/test/test_archive_getdate.c
(...)
/usr/src/usr.bin/tar/tests/Makefile
/usr/src/usr.sbin/freebsd-update/freebsd-update.sh
/usr/src/usr.sbin/rtsold/rtsol.c
Installing updates... done.
14.3-RELEASE-p7
[00:03:25] Recording filesystem state for clean... done
[00:03:25] Jail 14-3-R-amd64 14.3-RELEASE-p7 amd64 is ready to be used

test # poudriere jail -l
JAILNAME     VERSION         OSVERSION ARCH  METHOD TIMESTAMP           PATH
14-3-R-amd64 14.3-RELEASE-p7 1403000   amd64 http   2026-01-07 01:33:45 /usr/local/poudriere/jails/14-3-R-amd64

We also need FreeBSD Ports tree … just in a Poudriere way.

test # poudriere ports -c
[00:00:00] Creating default fs at /usr/local/poudriere/ports/default... done
[00:00:00] Cloning the ports tree...
fatal: unable to access 'https://git.FreeBSD.org/ports.git/': Could not resolve host: git.FreeBSD.org
[00:00:45] Error: /usr/local/share/poudriere/ports.sh:303: fail
[00:00:45] Error while creating ports tree, cleaning up.

This is where the dedicated git(1) config is needed as its a bitch and ignores *_PROXY variables 🙂

test # git config --system http.proxy http://10.0.0.41:3128

test # poudriere ports -c
[00:00:00] Creating default fs at /usr/local/poudriere/ports/default... done
[00:00:00] Cloning the ports tree... done

test # poudriere ports -l
PORTSTREE METHOD    TIMESTAMP           PATH
default   git+https 2026-01-07 01:49:46 /usr/local/poudriere/ports/default

Works.

Now – lets try to actually build something with Poudriere.

We will try two ports that one needs to be actually build (dosunix) and one that is in POSIX sh(1) and does not need building (lsblk).

I will intentionally run first building process with proxy variables disabled like this in the /usr/local/etc/poudriere.conf file:

# PACKAGE_FETCH_URL="http://pkg.FreeBSD.org/\${ABI}"
# PACKAGE_FETCH_BRANCH="latest"
# export HTTP_PROXY="http://10.0.0.41:3128"
# export HTTPS_PROXY="http://10.0.0.41:3128"
# export FTP_PROXY="http://10.0.0.41:3128"

Here.

test # poudriere bulk -c -C -j 14-3-R-amd64 -b latest -p default sysutils/lsblk converters/dosunix

Result is below … and as expected it failed.

test # poudriere bulk -c -C -j 14-3-R-amd64 -b latest -p default sysutils/lsblk converters/dosunix
[00:00:00] Creating the reference jail... done
[00:00:00] Mounting system devices for 14-3-R-amd64-default
[00:00:00] Stashing existing package repository
[00:00:00] Mounting ccache from: /var/ccache
[00:00:00] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:00] Mounting packages from: /usr/local/poudriere/data/packages/14-3-R-amd64-default
[00:00:00] Mounting distfiles from: /usr/ports/distfiles
[00:00:00] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/etc/resolv.conf
[00:00:00] Starting jail 14-3-R-amd64-default
Updating /var/run/os-release done.
[00:00:00] Will build as root:wheel (0:0)
[00:00:00] Ports supports: FLAVORS SUBPACKAGES SELECTED_OPTIONS
[00:00:00] Inspecting /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref//usr/ports for modifications to git checkout... no
[00:00:03] Ports top-level git hash: 284813ec0382a2bfe5b2e74a3081a67599d3155d
[00:00:03] Acquiring build logs lock for 14-3-R-amd64-default... done
[00:00:03] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_01h58m23s
[00:00:03] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_01h58m23s
[00:00:03] Loading MOVED for /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/usr/ports
[00:00:04] Gathering ports metadata
[00:00:04] Calculating ports order and dependencies
[00:00:04] Sanity checking the repository
[00:00:04] -c specified, cleaning all packages... done
[00:00:04] -C specified, cleaning listed packages
[00:00:04] (-C) Flushing package deletions
[00:00:04] Trimming IGNORED and blacklisted ports
[00:00:04] Package fetch: Looking for missing packages to fetch from pkg+http://pkg.FreeBSD.org/${ABI}/latest
[00:00:04] Package fetch: bootstrapping pkg
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest, please wait...
pkg: Attempted to fetch pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/Latest/pkg.pkg
pkg: Attempted to fetch pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/Latest/pkg.txz
pkg: Error: Address family for host not supported
Address resolution failed for http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest.
[00:08:58] Package fetch: Not fetching as remote repository is unavailable.
[00:08:58] pkg bootstrap missing: unable to inspect existing packages, cleaning all packages... done
[00:08:58] Deleting stale symlinks... done
[00:08:58] Deleting empty directories... done
[00:08:58] Unqueueing existing packages
[00:08:58] Unqueueing orphaned build dependencies
[00:08:58] Sanity checking build queue
[00:08:58] [14-3-R-amd64-default] [2026-01-07_01h58m23s] [pkgqueue_sanity_check] Time: 00:08:54
           Queued: 4 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched: 0 Remaining: 4
[00:08:58] Recording filesystem state for prepkg... done
[00:08:58] Processing PRIORITY_BOOST
[00:08:58] Building 4 packages using up to 4 builders
[00:08:58] Hit CTRL+t at any time to see build progress and stats
[00:08:58] [01] [00:00:00] Builder starting
[00:08:58] [01] [00:00:00] Builder started
[00:08:58] [01] [00:00:00] Building   ports-mgmt/pkg | pkg-2.5.1
[00:11:56] [01] [00:02:58] Finished   ports-mgmt/pkg | pkg-2.5.1: Failed: fetch
[00:11:56] [01] [00:02:58] Skipping   devel/ccache | ccache-3.7.12_8: Dependent port ports-mgmt/pkg | pkg-2.5.1 failed
[00:11:56] [01] [00:02:58] Skipping   converters/dosunix | dosunix-1.0.14: Dependent port ports-mgmt/pkg | pkg-2.5.1 failed
[00:11:56] [01] [00:02:58] Skipping   sysutils/lsblk | lsblk-4.0: Dependent port ports-mgmt/pkg | pkg-2.5.1 failed
[00:11:57] Stopping up to 4 builders
[00:11:57] Creating pkg repository
[00:11:57] No packages present
[00:11:57] Committing packages to repository: /usr/local/poudriere/data/packages/14-3-R-amd64-default/.real_1767751820 via .latest symlink
[00:11:57] Removing old packages
[00:11:57] Failed ports: ports-mgmt/pkg:fetch
[00:11:57] Skipped ports: converters/dosunix devel/ccache sysutils/lsblk
[00:11:57] [14-3-R-amd64-default] [2026-01-07_01h58m23s] [committing] Time: 00:11:53
           Queued: 4 Inspected: 0 Ignored: 0 Built: 0 Failed: 1 Skipped: 3 Fetched: 0 Remaining: 0
[00:11:57] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_01h58m23s
[00:11:57] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_01h58m23s
[00:11:57] Cleaning up
[00:11:57] Stopping up to 4 builders
[00:11:57] Unmounting file systems
test # 

Now I will run it again but with proxy setting enabled in the /usr/local/etc/poudriere.conf file like that.

# PACKAGE_FETCH_URL="http://pkg.FreeBSD.org/\${ABI}"
# PACKAGE_FETCH_BRANCH="latest"
export HTTP_PROXY="http://10.0.0.41:3128"
export HTTPS_PROXY="http://10.0.0.41:3128"
export FTP_PROXY="http://10.0.0.41:3128"

It should work with one small caveat.

test # poudriere bulk -c -C -j 14-3-R-amd64 -b latest -p default sysutils/lsblk converters/dosunix
[00:00:00] Creating the reference jail... done
[00:00:00] Mounting system devices for 14-3-R-amd64-default
[00:00:00] Stashing existing package repository
[00:00:00] Mounting ccache from: /var/ccache
[00:00:00] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:00] Mounting packages from: /usr/local/poudriere/data/packages/14-3-R-amd64-default
[00:00:00] Mounting distfiles from: /usr/ports/distfiles
[00:00:00] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/etc/resolv.conf
[00:00:00] Starting jail 14-3-R-amd64-default
Updating /var/run/os-release done.
[00:00:00] Will build as root:wheel (0:0)
[00:00:01] Ports supports: FLAVORS SUBPACKAGES SELECTED_OPTIONS
[00:00:01] Inspecting /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref//usr/ports for modifications to git checkout... no
[00:00:04] Ports top-level git hash: 284813ec0382a2bfe5b2e74a3081a67599d3155d 
[00:00:04] Acquiring build logs lock for 14-3-R-amd64-default... done
[00:00:04] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_02h13m56s
[00:00:04] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_02h13m56s
[00:00:04] Loading MOVED for /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/usr/ports
[00:00:04] Gathering ports metadata
[00:00:04] Calculating ports order and dependencies
[00:00:04] Sanity checking the repository
[00:00:04] -c specified, cleaning all packages... done
[00:00:04] -C specified, cleaning listed packages
[00:00:04] (-C) Flushing package deletions
[00:00:04] Trimming IGNORED and blacklisted ports
[00:00:04] Package fetch: Looking for missing packages to fetch from pkg+http://pkg.FreeBSD.org/${ABI}/latest
[00:00:04] Package fetch: bootstrapping pkg
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest, please wait...
[14-3-R-amd64-default] Installing pkg-2.5.1...
[14-3-R-amd64-default] Extracting pkg-2.5.1: 100%
Updating Poudriere repository catalogue...
pkg: No SRV record found for the repo 'Poudriere'
[14-3-R-amd64-default] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01    
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository Poudriere
Error updating repositories!
[00:00:29] Package fetch: Not fetching as remote repository is unavailable.
[00:00:29] pkg bootstrap missing: unable to inspect existing packages, cleaning all packages... done
[00:00:29] Deleting stale symlinks... done
[00:00:29] Deleting empty directories... done
[00:00:29] Unqueueing existing packages
[00:00:29] Unqueueing orphaned build dependencies
[00:00:29] Sanity checking build queue
[00:00:29] [14-3-R-amd64-default] [2026-01-07_02h13m56s] [pkgqueue_sanity_check] Time: 00:00:26
           Queued: 4 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched: 0 Remaining: 4
[00:00:29] Recording filesystem state for prepkg... done
[00:00:29] Processing PRIORITY_BOOST
[00:00:30] Building 4 packages using up to 4 builders
[00:00:30] Hit CTRL+t at any time to see build progress and stats
[00:00:30] [01] [00:00:00] Builder starting
[00:00:30] [01] [00:00:00] Builder started
[00:00:30] [01] [00:00:00] Building   ports-mgmt/pkg | pkg-2.5.1
[00:02:34] [01] [00:02:04] Finished   ports-mgmt/pkg | pkg-2.5.1: Success
[00:02:34] [02] [00:00:00] Builder starting
[00:02:34] [01] [00:00:00] Building   devel/ccache | ccache-3.7.12_8
[00:02:35] [02] [00:00:01] Builder started
[00:02:35] [02] [00:00:00] Building   sysutils/lsblk | lsblk-4.0
[00:02:36] [02] [00:00:01] Finished   sysutils/lsblk | lsblk-4.0: Success
[00:02:39] [01] [00:00:05] Finished   devel/ccache | ccache-3.7.12_8: Success
[00:02:39] [01] [00:00:00] Building   converters/dosunix | dosunix-1.0.14
[00:02:42] [01] [00:00:03] Finished   converters/dosunix | dosunix-1.0.14: Success
[00:02:42] Stopping up to 4 builders
[00:02:42] Creating pkg repository
[00:02:42] Signing repository with key: /usr/local/etc/ssl/keys/poudriere.key
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:02:43] Signing pkg bootstrap with method: pubkey
[00:02:43] Committing packages to repository: /usr/local/poudriere/data/packages/14-3-R-amd64-default/.real_1767752199 via .latest symlink
[00:02:43] Removing old packages
[00:02:43] Built ports: ports-mgmt/pkg sysutils/lsblk devel/ccache converters/dosunix
[00:02:43] [14-3-R-amd64-default] [2026-01-07_02h13m56s] [committing] Time: 00:02:39
           Queued: 4 Inspected: 0 Ignored: 0 Built: 4 Failed: 0 Skipped: 0 Fetched: 0 Remaining: 0
[00:02:43] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_02h13m56s
[00:02:43] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_02h13m56s
[00:02:43] Cleaning up
[00:02:43] Stopping up to 4 builders
[00:02:43] Unmounting file systems

Also in nice graphical colored form.

The build generally ended successfully – we have our packages available.

test # ls -l /usr/local/poudriere/data/packages/14-3-R-amd64-default/All/
total 6294
-rw-r--r--  1 root wheel  126041 Jan  7 02:16 ccache-3.7.12_8.pkg
-rw-r--r--  1 root wheel    5963 Jan  7 02:16 dosunix-1.0.14.pkg
-rw-r--r--  1 root wheel    6544 Jan  7 02:16 lsblk-4.0.pkg
-rw-r--r--  1 root wheel 6290261 Jan  7 02:16 pkg-2.5.1.pkg

The only errors were these below and they did not broken our build:

(...)
Updating Poudriere repository catalogue...
pkg: No SRV record found for the repo 'Poudriere'
[14-3-R-amd64-default] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01    
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/data.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/data.tzst -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest/packagesite.tzst -- pkg+:// implies SRV mirror type
Unable to update repository Poudriere
Error updating repositories!
[00:00:29] Package fetch: Not fetching as remote repository is unavailable.
(...)

To get the idea what is wrong here we need to level up our debugging skill and get our hands dirty with FreeBSD tools like ktrace(8) and kdump(8) to know what is missing.

test # ktrace -di poudriere bulk -c -C -j 14-3-R-amd64 -b latest -p default sysutils/lsblk converters/dosunix
(...)
[00:00:32] [01] [00:00:00] Builder starting
[00:00:33] [01] [00:00:01] Builder started
[00:00:33] [01] [00:00:00] Building   ports-mgmt/pkg | pkg-2.5.1

When You reach this place – just hit [CTRL]+[C] to stop it – its not needed to wait for it.

Now check with kdump(8) the gathered data.

test # kdump | grep -m 5 -C 2 'pkg+' | tail -4
        Poudriere: {
        url: pkg+http://pkg.FreeBSD.org/${ABI}/latest,
        mirror_type: srv
        }

I wanted to show only the part that was important – Of course I did not guessed it like that … just find it with a help of a friend.

Poudriere – on the fly – defines additional repository … and if you did not override it – yes – it will use both pkg+ and srv things that hurt our proxy environment.

This is part of the Poudriere code that is responsible for its generation.

cat >> "${MASTERMNT:?}/etc/pkg/poudriere.conf" <<-EOF
FreeBSD: {
        enabled: no,
        priority: 100
}
FreeBSD-kmods: {
        enabled: no,
        priority: 100
}
FreeBSD-ports: {
        enabled: no,
        priority: 100
}
FreeBSD-ports-kmods: {
        enabled: no,
        priority: 100
}
FreeBSD-base: {
        enabled: no,
        priority: 100
}

Poudriere: {
        url: ${packagesite},
        mirror_type: $(if [ "${packagesite#pkg+}" = "${packagesite}" ]; then echo "none"; else echo "srv"; fi)
}
EOF

So it will generate the ‘broken for proxy’ config like that:

Poudriere: {
        url: pkg+http://pkg.FreeBSD.org/${ABI}/latest,
        mirror_type: srv
}

Looking at the code above you can see the if that checks how the packagesite is defined.

Lets see how Poudriere figures that one out in the code.

test # grep -m 1 packagesite= common.sh
        packagesite="${PACKAGE_FETCH_URL:+${PACKAGE_FETCH_URL}/}${PACKAGE_FETCH_BRANCH}"

So the answer is that we need to set PACKAGE_FETCH_URL and PACKAGE_FETCH_BRANCH in the /usr/local/etc/poudriere.conf file … the ones I commented out to explain all that in details.

So now – with all needed settings enabled at /usr/local/etc/poudriere.conf file … fully working Poudriere build in proxy environment.

PACKAGE_FETCH_URL="http://pkg.FreeBSD.org/\${ABI}"
PACKAGE_FETCH_BRANCH="latest"
export HTTP_PROXY="http://10.0.0.41:3128"
export HTTPS_PROXY="http://10.0.0.41:3128"
export FTP_PROXY="http://10.0.0.41:3128"

Remember to have the \$ as backslashed as Poudriere is written in POSIX sh(1).

Now the fully working run.

test # poudriere bulk -c -C -j 14-3-R-amd64 -b latest -p default sysutils/lsblk converters/dosunix
[00:00:00] Creating the reference jail... done
[00:00:00] Mounting system devices for 14-3-R-amd64-default
[00:00:00] Stashing existing package repository
[00:00:00] Mounting ccache from: /var/ccache
[00:00:00] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:00] Mounting packages from: /usr/local/poudriere/data/packages/14-3-R-amd64-default
[00:00:00] Mounting distfiles from: /usr/ports/distfiles
[00:00:00] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/etc/resolv.conf
[00:00:00] Starting jail 14-3-R-amd64-default
Updating /var/run/os-release done.
[00:00:00] Will build as root:wheel (0:0)
[00:00:00] Ports supports: FLAVORS SUBPACKAGES SELECTED_OPTIONS
[00:00:00] Inspecting /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref//usr/ports for modifications to git checkout... no
[00:00:03] Ports top-level git hash: 284813ec0382a2bfe5b2e74a3081a67599d3155d 
[00:00:03] Acquiring build logs lock for 14-3-R-amd64-default... done
[00:00:03] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_03h52m21s
[00:00:03] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_03h52m21s
[00:00:03] Loading MOVED for /usr/local/poudriere/data/.m/14-3-R-amd64-default/ref/usr/ports
[00:00:04] Gathering ports metadata
[00:00:04] Calculating ports order and dependencies
[00:00:04] Sanity checking the repository
[00:00:04] -c specified, cleaning all packages... done
[00:00:04] -C specified, cleaning listed packages
[00:00:04] (-C) Flushing package deletions
[00:00:04] Trimming IGNORED and blacklisted ports
[00:00:04] Package fetch: Looking for missing packages to fetch from http://pkg.FreeBSD.org/${ABI}/latest
[00:00:04] Package fetch: bootstrapping pkg
Bootstrapping pkg from http://pkg.FreeBSD.org/FreeBSD:14:amd64/latest, please wait...
[14-3-R-amd64-default] Installing pkg-2.5.1...
[14-3-R-amd64-default] Extracting pkg-2.5.1: 100%
Updating Poudriere repository catalogue...
[14-3-R-amd64-default] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01    
[14-3-R-amd64-default] Fetching data: 100%   11 MiB   3.7 M/s    00:03    
Processing entries: 100%
Poudriere repository update completed. 36661 packages processed.
All repositories are up to date.
[00:00:20] Package fetch: Will fetch 3 packages from remote or local pkg cache
Updating database digests format: 100%
The following packages will be fetched:

New packages to be FETCHED:
        ccache: 3.7.12_8 (134 KiB: 91.17% of the 147 KiB to download)
        dosunix: 1.0.14 (6 KiB: 4.04% of the 147 KiB to download)
        lsblk: 4.0 (7 KiB: 4.79% of the 147 KiB to download)

Number of packages to be fetched: 3

147 KiB to be downloaded.
[14-3-R-amd64-default] Fetching ccache-3.7.12_8: 100%  134 KiB 137.3 k/s    00:01    
[14-3-R-amd64-default] Fetching lsblk-4.0: 100%    7 KiB   7.2 k/s    00:01    
[14-3-R-amd64-default] Fetching dosunix-1.0.14: 100%    6 KiB   6.1 k/s    00:01    
[00:00:21] Package fetch: Using cached copy of ccache-3.7.12_8
[00:00:21] Package fetch: Using cached copy of dosunix-1.0.14
[00:00:21] Package fetch: Using cached copy of lsblk-4.0
[00:00:21] Checking packages for incremental rebuild needs
[00:00:21] Deleting stale symlinks... done
[00:00:21] Deleting empty directories... done
[00:00:21] Package fetch: Generating logs for fetched packages
[00:00:21] Unqueueing existing packages
[00:00:21] Unqueueing orphaned build dependencies
[00:00:21] Sanity checking build queue
[00:00:21] [14-3-R-amd64-default] [2026-01-07_03h52m21s] [pkgqueue_sanity_check] Time: 00:00:18
           Queued: 4 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched: 3 Remaining: 1
[00:00:21] Recording filesystem state for prepkg... done
[00:00:21] Processing PRIORITY_BOOST
[00:00:21] Building 1 packages using up to 1 builders
[00:00:21] Hit CTRL+t at any time to see build progress and stats
[00:00:21] [01] [00:00:00] Builder starting
[00:00:21] [01] [00:00:00] Builder started
[00:00:21] [01] [00:00:00] Building   ports-mgmt/pkg | pkg-2.5.1
[00:02:25] [01] [00:02:04] Finished   ports-mgmt/pkg | pkg-2.5.1: Success
[00:02:25] Stopping up to 1 builders
[00:02:25] Creating pkg repository
[00:02:25] Signing repository with key: /usr/local/etc/ssl/keys/poudriere.key
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:02:25] Signing pkg bootstrap with method: pubkey
[00:02:25] Committing packages to repository: /usr/local/poudriere/data/packages/14-3-R-amd64-default/.real_1767758087 via .latest symlink
[00:02:25] Removing old packages
[00:02:25] Built ports: ports-mgmt/pkg
[00:02:25] Fetched ports: sysutils/lsblk converters/dosunix devel/ccache
[00:02:25] [14-3-R-amd64-default] [2026-01-07_03h52m21s] [committing] Time: 00:02:23
           Queued: 4 Inspected: 0 Ignored: 0 Built: 1 Failed: 0 Skipped: 0 Fetched: 3 Remaining: 0
[00:02:25] Logs: /usr/local/poudriere/data/logs/bulk/14-3-R-amd64-default/2026-01-07_03h52m21s
[00:02:25] WWW: http://0.0.0.0//build.html?mastername=14-3-R-amd64-default&build=2026-01-07_03h52m21s
[00:02:25] Cleaning up
[00:02:25] Stopping up to 4 builders
[00:02:25] Unmounting file systems
test # 

… and in the TECHNICOLOR form 🙂

The part that was broken earlier is now fine.

Updating Poudriere repository catalogue...
[14-3-R-amd64-default] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01    
[14-3-R-amd64-default] Fetching data: 100%   11 MiB   3.7 M/s    00:03    
Processing entries: 100%

I believe that concludes this article – let me know if I missed anything.

EOF
Top

2025: A Year of Advocacy, Community, and Growth

Post by FreeBSD Foundation via FreeBSD Foundation »

Engagement has been outstanding, and the overwhelmingly positive feedback reinforces what we’re seeing elsewhere: interest in FreeBSD is on the rise.

As we reflect on 2025, it is clear that this has been a transformative year for the FreeBSD Project and the FreeBSD Foundation. From expanding educational outreach and strengthening our social presence to fostering deeper connections across the global community, we continued to advance our mission of supporting FreeBSD through advocacy, visibility, and community engagement.

Thanks to the generosity of our donors and the dedication of contributors, partners, and staff, we delivered meaningful impact across advocacy, marketing, events, and collaboration. Below are the key highlights that shaped our advocacy and community efforts in 2025. 

Advocacy & Community Engagement

Throughout 2025, the Foundation expanded its role in promoting FreeBSD and supporting community growth. Our work centered on making FreeBSD more visible, accessible, and engaging for both new and experienced users.

The 2025 Community Survey, sponsored jointly by the FreeBSD Core Team and the Foundation, provided valuable insights into user needs, particularly regarding onboarding, documentation, and accessibility, which helped shape our advocacy and educational priorities for the year.

To strengthen authentic community engagement, we encouraged users to share their FreeBSD success stories through blog submissions and social channels. This storytelling helped highlight FreeBSD’s value across research, industry, and education.

A standout example was Leah Budzicka, a student who discovered the Foundation through our social media and later attended EuroBSDCon through our Travel Grant Program:
https://freebsdfoundation.org/blog/2025-eurobsdcon-trip-report-leah-budzicka/

These trip reports continue to illustrate the real-world impact of community support, mentorship, and Foundation-funded programs.

Events & Global Engagement

Throughout the year, the Foundation participated in major events across the open-source and BSD ecosystems. These gatherings strengthened relationships across the Project, provided spaces for technical collaboration, and allowed us to share FreeBSD updates with global audiences.

Key 2025 events included:

  • FreeBSD Developer & Vendor Summits — San Jose, CA
  • EuroBSDCon 2025 — Zagreb, Croatia
  • EuroBSDCon Developer Summit — Zagreb, Croatia
  • Open Source Summit North America — Denver, CO
  • BSDCan 2025 — Ottawa, Canada

A full list of events can be viewed here:
https://freebsdfoundation.org/our-work/events/past-events/

These events provided opportunities to highlight key FreeBSD Project work, including progress on FreeBSD 15, CI/CD improvements, CHERI integration, Sovereign Tech Agency collaborations, and Google Summer of Code achievements, while also supporting students, new contributors, and long-time community members.

Why This Matters

Our advocacy and community programs play a critical role in ensuring FreeBSD remains visible, inclusive, and welcoming. By listening to user needs, amplifying community stories, and showing up at events around the world, we help grow a stronger, more connected FreeBSD ecosystem.

Marketing & Advocacy: Growing the Next Generation

2025 marked a year of growth in educational outreach, communications, and global engagement. Through sustained advocacy work, we continued expanding visibility for FreeBSD and strengthening connections within the community.

Expanding Educational Content & Visibility

Across YouTube, LinkedIn, Facebook, Mastodon, Instagram, and Bluesky, we published educational content, technical explainers, interviews, and event highlights to help users learn, adopt, and explore FreeBSD.

A major milestone was the restructuring of our YouTube channel and a focus on high-production-quality technical content.

By separating meeting recordings out into a dedicated “FreeBSD Meetings” channel, we helped the YouTube algorithm to ultimately give a better experience for our viewers.

Together, these efforts helped the main FreeBSD YouTube channel grow by more than 3,000 new subscribers this year, a 50% increase. Engagement has been outstanding, and the overwhelmingly positive feedback reinforces what we’re seeing elsewhere: interest in FreeBSD is on the rise.

Additionally, we continued developing blogs, videos, and long-term learning resources designed to onboard the next generation of FreeBSD contributors.

 

Social Growth & Top Content of the Year 

Top Blogs of 2025:

Top YouTube Videos of 2025:

In 2025, we used social media to tell the FreeBSD story, highlighting community voices, technical innovation, and the work powering the project forward. Through consistent storytelling across LinkedIn and YouTube, we expanded our reach, strengthened community engagement, and brought greater visibility to technical milestones, educational content, and contributor voices. These efforts helped grow our audience, drive meaningful discussion, and connect more developers and users to the FreeBSD ecosystem. A detailed breakdown of metrics, growth, and platform performance is available in our full Social Media Year-End Report.

Corporate Donor Highlights

We are deeply grateful for the organizations that invest in FreeBSD’s future through annual contributions and sponsorships. Their support directly funds software development, infrastructure improvements, community programs, and global advocacy.

If your company is not yet a donor but recognizes the importance of investing in FreeBSD, we invite you to explore our partnership program.

A full list of 2025 supporters can be viewed here:
https://freebsdfoundation.org/donors/

Community Appreciation & Sustainable Open Source Stewardship

This year reinforced an essential truth across open source: meaningful progress only happens when communities, contributors, and organizations share responsibility for sustaining the work that supports us all.

The FreeBSD community continues to demonstrate what collaborative stewardship looks like. From maintainers and developers to documentation writers, testers, advocates, educators, sponsors, and donors, every contribution helps move the Project forward.

We are grateful for the bug reports, code reviews, travel grant stories, conference talks, documentation improvements, tutorials, and moments of mentorship that shaped 2025. Your support—whether financial, technical, or community-driven—ensures that FreeBSD remains modern, stable, and accessible to everyone who relies on it.

Thank you for helping build a stronger, more resilient, and more sustainable FreeBSD ecosystem.

 

Looking Ahead to 2026

As we enter 2026, the Foundation will continue investing in high-impact development, improving the contributor experience, strengthening global infrastructure, and expanding educational programs that empower both new and experienced users.

If you’d like to invest in our efforts, please consider donating to the Foundation. Your support, no matter the size, makes a direct impact on the work we do for the Project. Together, we are building the long-term future of FreeBSD, and we are grateful to everyone who helps make this work possible.

Donate to FreeBSD Foundation | FreeBSD Foundation

 

Written by Florine Kamdem

 

The post 2025: A Year of Advocacy, Community, and Growth first appeared on FreeBSD Foundation.

Top

Installing and using SearXNG in Gentoo Linux

Post by Nathan Zachary via The Z-Issue »

Introduction

Searching the internet has been both increasingly cumbersome and frightening due to privacy concerns with many of the major search engine providers (such as Google or Bing). Thankfully there are some other good options like Startpage, Qwant, and Mojeek. Some of them use results provided by the major providers like Google, and some of them are more independent. However, another option exists, and that’s to use an application like SearXNG that acts as a meta-search engine, “aggregating the results of other search engines” without “storing information about its users”.

I started using SearXNG quite some time ago by choosing a public instance, but have recently gone a bit further and installed my own self-hosted version of it on one of my Gentoo Linux servers. In this article, I’m going to outline the process of installing, configuring, and running a self-hosted SearXNG instance along with some troubleshooting tips and “gotchas” to avoid. Though the details are specific to Gentoo Linux, the concepts should be readily applicable to other Linux distributions.

For my setup, I chose to install SearXNG in the uWSGI application server and proxy it via the Apache web server using the mod_proxy_uwsgi module. The flow of data follows the following design:

For the remainder of this article, I will use the pretend domain name of my-secret-search-engine.com. Any place where that domain is listed, replace it with your respective domain name or IP address of your server.

Installation – Prerequisites

To start, SearXNG should run as a system user, so that user should be created accordingly:

useradd -s /bin/bash --system -m --home-dir /usr/local/searxng searxng

Now switch to that new user and create a Python virtual environment (venv) for installing and running the application and all of its dependencies. The Python venv is isolated from the overall system installation of Python so that there aren’t any conflicts with the system’s Python interpreter or module versions.

su - searxng
python -m venv /usr/local/searxng/searxng-pyvenv

The Python virtual environment needs to be activated in order to use it. It’s helpful to automatically activate it when switching to the ‘searxng’ user, which can be done by putting it in that user’s .bashrc:

echo "source /usr/local/searxng/searxng-pyvenv/bin/activate" >> /usr/local/searxng/.bashrc

To make sure that the automatic activation is working as intended, switch back to the root account and then back to the ‘searxng’ user:

exit
su - searxng

If the venv is activated, the name of the venv should be displayed before the standard BASH prompt:

# su - searxng
Last login: Thu Oct 24 14:52:00 UTC 2025 on pts/1
(searxng-pyvenv) searxng@my-secret-search ~ $

SearXNG has some prerequisite Python modules that need to be installed, so install them using Python’s own package manager, Pip:

pip install --upgrade pip setuptools wheel pyyaml msgspec

To see the modules and versions that are currently installed in the virtual environment, the freeze subcommand can be issued in Pip:

$ pip freeze
PyYAML==6.0.3
setuptools==80.9.0
wheel==0.45.1
msgspec==0.19.0

Installation – SearXNG application

SearXNG doesn’t currently offer packaged “releases”, so the way to install it is to clone their Git repository and build the application from that clone:

mkdir -p /usr/local/searxng/searxng-pyvenv/source/
git clone "https://github.com/searxng/searxng" "/usr/local/searxng/searxng-pyvenv/source"
cd /usr/local/searxng/searxng-pyvenv/source/
pip install --use-pep517 --no-build-isolation -e .

After the installation finishes, there will be many more modules listed in Pip (the exact modules and versions may change with subsequent SearXNG versions):

$ pip freeze
anyio==4.11.0
async-timeout==5.0.1
babel==2.17.0
blinker==1.9.0
Brotli==1.1.0
certifi==2025.10.5
click==8.3.0
fasttext-predict==0.9.2.4
Flask==3.1.2
flask-babel==4.0.0
h11==0.16.0
h2==4.3.0
hpack==4.1.0
httpcore==1.0.9
httpx==0.28.1
httpx-socks==0.10.0
hyperframe==6.1.0
idna==3.11
isodate==0.7.2
itsdangerous==2.2.0
Jinja2==3.1.6
lxml==6.0.2
markdown-it-py==3.0.0
MarkupSafe==3.0.3
mdurl==0.1.2
msgspec==0.19.0
Pygments==2.19.2
python-dateutil==2.9.0.post0
python-socks==2.7.2
pytz==2025.2
PyYAML==6.0.3
setproctitle==1.3.7
setuptools==80.9.0
six==1.17.0
sniffio==1.3.1
typer-slim==0.19.2
typing_extensions==4.14.1
valkey==6.1.1
Werkzeug==3.1.3
wheel==0.45.1
whitenoise==6.11.0

Installation – uWSGI application server

Now that the SearXNG application itself has been installed, it’s time to install the application server (before proceeding to configure both the application server and SearXNG itself). I chose to go with uWSGI. Since uWSGI supports many different application languages and various plugins, and seeing as they can be easily configured with Gentoo’s USE flags, they should be set before installing uWSGI:

# grep uwsgi /etc/portage/package.use 
www-servers/uwsgi -embedded -php python UWSGI_PLUGINS: -carbon -http -mongodblog -nagios -redislog -rrdtool pam

# emerge -av uwsgi

With the uWSGI server installed, it can be started automatically at boot time using Gentoo’s OpenRC init scripts:

cd /etc/conf.d
cp -v uwsgi uwsgi.searxng
ln -s /etc/init.d/uwsgi{,.searxng}
rc-update add uwsgi.searxng default

The above commands make a copy of the uWSGI configuration file specifically for SearXNG, and then create a new symlink of the uWSGI init script as /etc/init.d/uwsgi.searxng. Lastly, that new uwsgi.searxng init script is started at the default runlevel during the boot process.

Configuration – uWSGI application server

With both the SearXNG application and the uWSGI application server installed, it’s time to configure both of them accordingly. We’ll start with the uWSGI application server as it is more straightforward. First, we will copy the uWSGI template provided by SearXNG to the overall system location:

cp -v /usr/local/searxng/searxng-pyvenv/source/utils/templates/etc/uwsgi/apps-available/searxng.ini /etc/searxng/searxng.ini

Second, we will edit the uWSGI startup configuration to call this file:

echo 'UWSGI_EXTRA_OPTIONS="--ini /etc/searxng/searxng.ini"' >> /etc/conf.d/uwsgi.searxng

The full configuration file (excluding comments and blank lines) should look similar to the output below. The only other change that I made was to add a path for the log file. As such, the lines that I changed are in purple italics here, but feel free to make other changes for your particular setup or needs:

# grep -v "^#\|^$" /etc/conf.d/uwsgi.searxng 
UWSGI_SOCKET=
UWSGI_THREADS=0
UWSGI_PROGRAM=
UWSGI_XML_CONFIG=
UWSGI_PROCESSES=1
UWSGI_LOG_FILE=/var/log/uwsgi.log
UWSGI_CHROOT=
UWSGI_DIR=
UWSGI_PIDPATH_MODE=0750
UWSGI_USER=
UWSGI_GROUP=
UWSGI_EMPEROR_PATH=
UWSGI_EMPEROR_PIDPATH_MODE=0770
UWSGI_EMPEROR_GROUP=
UWSGI_EXTRA_OPTIONS="--ini /etc/searxng/searxng.ini"

Configuration – SearXNG application

With uWSGI being configured, it’s time to configure the SearXNG application itself. The configurations for it are handled by two separate files:

  • /etc/searxng/searxng.ini –> settings for running SearXNG as an application in uWSGI
  • /etc/searxng/settings.yml –> configuration options for the application itself

First, make the directory for storing these two settings files and copy them from the SearXNG sources:

mkdir -p /etc/searxng/
cp -v /usr/local/searxng/searxng-pyvenv/source/utils/templates/etc/uwsgi/apps-available/searxng.ini /etc/searxng/
cp -v /usr/local/searxng/searxng-pyvenv/source/utils/templates/etc/searxng/settings.yml /etc/searxng/
There are two versions of the settings.yml file that can be used:

/usr/local/searxng/searxng-pyvenv/source/utils/templates/etc/searxng/settings.yml
OR
/usr/local/searxng/searxng-pyvenv/source/searx/settings.yml

The first one (and the one used here) is much more compact and basically sets the defaults for most things, allowing for overrides where desired. I would strongly suggest using this first template unless you have a very specific reason to use the full one.

Starting with /etc/searxng/searxng.ini (which again is for configuring how the application will run inside the uWSGI server), here are the settings I suggest (pay close attention to the settings in purple italics as they are pertinent):

# grep -v "^#\|^$" /etc/searxng/searxng.ini 
[uwsgi]
uid = searxng
gid = searxng
env = LANG=C.UTF-8
env = LANGUAGE=C.UTF-8
env = LC_ALL=C.UTF-8
chdir = /usr/local/searxng/searxng-pyvenv/source/searx/ 
env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
disable-logging = true
chmod-socket = 666
single-interpreter = true
master = true
lazy-apps = true
plugin = python313,asyncio313
enable-threads = true
workers = ${UWSGI_WORKERS:-%k}
threads = ${UWSGI_THREADS:-4}
module = searx.webapp
virtualenv = /usr/local/searxng/searxng-pyvenv
pythonpath = /usr/local/searxng/searxng-pyvenv/source/searxng
socket = /usr/local/searxng/run/socket
buffer-size = 8192
offload-threads = %k

Now for the SearXNG settings themselves (in /etc/searxng/settings.yml), the template yields the following options by default:

# grep -v "^#\|^  #\|^$" /usr/local/searxng/searxng-pyvenv/source/utils/templates/etc/searxng/settings.yml
use_default_settings: true
general:
  debug: false
  instance_name: "SearXNG"
search:
  safe_search: 2
  autocomplete: 'duckduckgo'
  formats:
    - html
server:
  secret_key: "ultrasecretkey"
  limiter: true
  image_proxy: true
valkey:
  url: valkey://localhost:6379/0

The only option that MUST be changed is the ‘secret_key’, and it can be done easily with:

sed -i -e "s/ultrasecretkey/$(openssl rand -hex 16)/g" /etc/searxng/settings.yml

There are many other options that can be set or unset, and they are fairly well documented. Here’s an example settings.yml file with some of the settings that I personally prefer, followed by a brief explanation of them:

# grep -v "^#\|^  #\|^$" /etc/searxng/settings.yml 
use_default_settings: true
general:
  debug: false
  instance_name: "My Secret Search Engine"
  enable_metrics: false
search:
  safe_search: 0
  autocomplete: ""
  formats:
    - html
server:
  secret_key: "YOUR_RANDOM_KEY"
  limiter: false
  image_proxy: true
  http_protocol_version: "1.1"
  method: "GET"
ui:
  theme_args:
    simple_style: black
  url_formatting: full
OptionExplanation
instance_name:Setting it to your site's name will display it in the browser's title bar
enable_metrics: falseVarious anonymous metrics; disabled for even better anonymity
safe_search: 0Disables all "safe search" filtering
autocomplete: ""Which autocomplete backend to use; in this case none
image_proxy: trueUses your instance to proxy images for better anonymity
method: "GET"Favours some ease of use over the more private "POST"
url_formatting: fullShows the full URL of all links instead of a breadcrumb ("pretty")
Pay special attention to the ‘use_default_settings:’ declaration. I have included some additional syntax information about it in the “Troubleshooting and Gotchas” section at the bottom of the article.

Configuration – Apache web server

Now that both the uWSGI application server and the SearXNG application itself have been installed and configured, the last step is to proxy through the Apache web server.

Though uWSGI can be set up to handle HTTP requests directly—thus removing the need for Apache or a different web server—I prefer to keep it as a backend and let a dedicated web server manage HTTP connections.

Doing so requires two Apache modules: mod_proxy and mod_proxy_uwsgi. Linux distributions have different methods of enabling Apache modules, so consult your distribution’s documentation on doing so. In Gentoo, it is done by adding “proxy” and “proxy_uwsgi” to APACHE2_MODULES and re-emerging it.

It’s also important to load mod_proxy BEFORE mod_proxy_uwsgi, so make sure that the order is correct in the module-loading section of /etc/apache2/httpd.conf:

# grep proxy /etc/apache2/httpd.conf 
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so

As for the Apache vhost configuration, it is likely quite similar to any other site you have configured (and generic vhost configuration is outside the scope of this article). The parts that are specific to SearXNG are the proxy directives. Assuming that SearXNG is going to be accessed at the root of the domain, the proxy directives should look like:

## Removed these main directives from <Location> block for Certbot.
## This allows bypassing the proxy for the .well-known/ subdirectory for LetsEncrypt
        ProxyPreserveHost On
        ProxyPass /.well-known/ !
        ProxyPass / unix:/usr/local/searxng/run/socket|uwsgi://localhost/

        RequestHeader set X-Forwarded-Proto %{REQUEST_SCHEME}s
        RequestHeader set X-Script-Name /searxng
        RequestHeader set X-Real-IP %{REMOTE_ADDR}s
        RequestHeader append X-Forward-For %{REMOTE_ADDR}s

The ‘ProxyPass’ line in purple is the one that is responsible for proxying the HTTP request via UNIX socket to the uWSGI application server and vice-versa. After reloading Apache, your SearXNG instance should now be accessible. If it isn’t, you may find some additional pointers in the “Troubleshooting and Gotchas” section at the bottom of this article.

20260106 Update:
In the Apache vhost configuration above, I removed the whole block from the <Location> tags so that I could add in a bypass for the proxy (the line in orange). This allows for Certbot to renew the SSL certificate since that particular subdirectory is excluded from the proxied uWSGI application.

Updating the SearXNG application

As previously mentioned, SearXNG doesn’t publish releases so updating involves pulling down the latest Git master branch, updating dependencies, and then rebuilding. There are many different methods for performing these updates in Git, but my approach is:

su - searxng
cd /usr/local/searxng/searxng-pyvenv/source
git pull origin master
pip install --upgrade pip setuptools wheel pyyaml msgspec
pip install --upgrade --use-pep517 --no-build-isolation -e .
exit
/etc/init.d/uwsgi.searxng restart

The steps outlined in the code block above will:

  • Switch to the ‘searxng’ user, which will activate the Python virtual environment (assuming you followed the instructions in the ‘Installation – Prerequisites’ section of this article)
  • Change to the SearXNG source directory (Git)
  • Pull the latest files from SearXNG’s Git master branch
  • Use pip to update the dependencies
  • Use pip to rebuild the application using the latest sources
  • Switch back from the ‘searxng’ user back to root (“exit”)
  • Restart the uWSGI application server containing the SearXNG application

After these steps, SearXNG should be the latest version. This can be validated by checking the following line in the footer of the page. For example:

Powered by SearXNG – 2025.10.24+2cdbbb249

That line should show the date of the pull and the first 9 characters of the latest commit that was pulled.

Updating – Rolling back

In case a particular update causes a problem, it can be readily rolled back to the previous version you had installed by performing a git reset:

su - searxng
cd /usr/local/searxng/searxng-pyvenv/source
git reset --keep HEAD@{1}
pip install --upgrade --use-pep517 --no-build-isolation -e .
exit
/etc/init.d/uwsgi.searxng restart

Troubleshooting and Gotchas

If you run into problems with the installation or configuration steps throughout this article, some of the more common errors and pitfalls are listed here, separated into sections based on where the problem lies.

SearXNG application

The SearXNG settings are documented quite well, but I did run into a particular syntax discrepancy that caused unexpected results. When using the use_default_settings directive, there is a syntax difference based on whether or not any of the search engines will be modified.

If no engine modifications will be made:

WORKS:

use_default_settings: true

FAILS:

use_default_settings:

If, however, any engine modifications are present, the true value must be dropped.

WORKS:

use_default_settings:
  engines:
    remove:
      - google

FAILS:

use_default_settings: true
  engines:
    remove:
      - google

In the ‘fails’ scenarios, the uWSGI log will show errors related to various Python scripts with these lines being at or near the end of the trace:

File "/usr/local/searxng/searxng-pyvenv/source/searx/settings_loader.py", line 218, in load_settings
user_cfg = load_yaml(cfg_file)
File "/usr/local/searxng/searxng-pyvenv/source/searx/settings_loader.py", line 48, in load_yaml
raise SearxSettingsException(e, str(file_name)) from e
searx.exceptions.SearxSettingsException: mapping values are not allowed here
in "/etc/searxng/settings.yml", line 5, column 10
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***

uWSGI application server

If the following error message appears in the uWSGI log:

bind(): No such file or directory [core/socket.c line 230]

it is likely that the path to the UNIX socket is not present or that the permissions are incorrect. Check the ‘socket =’ line in /etc/searxng/searxng.ini and make sure the path is there and that the permissions are 755 owned by the ‘searxng’ user:

# ls -lhd /usr/local/searxng/run/
drwxr-xr-x 2 searxng searxng 4.0K Oct 24 17:38 /usr/local/searxng/run/

Apache web server

Though the Apache vhost configuration for SearXNG is essentially just the proxy section listed above, there are many syntax errors that can arise from it. The ProxyPass line—which passes HTTP requests to the UNIX socket—can be particularly finicky about syntax:

ProxyPass unix:/usr/local/searxng/run/socket|uwsgi://localhost/

I found that only specifying the ‘uwsgi://’ protocol resulted in the application failing. In my case, I needed it to specifically reference ‘localhost’. As such, the following ProxyPass directive did not work for me:

ProxyPass unix:/usr/local/searxng/run/socket|uwsgi://
Top

Valuable News – 2026/01/05

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD vs Linux – Which is Better.
https://netdata.cloud/academy/freebsd-vs-linux/

Infrastructure Modernization – Commissioned by Sovereign Tech Agency.
https://freebsdfoundation.org/blog/infrastructure-modernization-commissioned-by-the-sovereign-tech-agency/

OpenBSD Stories – When Driver Challenges Kernel Assumptions.
http://miod.online.fr/software/openbsd/stories/udl.html

FreeBSD Git Weekly: 2025-12-22 to 2025-12-28.
https://freebsd-git-weekly.tarsnap.net/2025-12-22.html

Set It and Forget It – Monit on FreeBSD in Minutes.
https://youtube.com/watch?v=ZkM228fbykU

39C3 – Escaping Containment – Security Analysis of FreeBSD Jails.
https://media.ccc.de/v/39c3-escaping-containment-a-security-analysis-of-freebsd-jails

HardenedBSD 2025/12 Status Report.
https://hardenedbsd.org/article/shawn-webb/2025-12-28/hardenedbsd-december-2025-status-report

OpenBSD fw_update(8) Now Checks dmesg(8) Output in Addition to dmesg.boot File.
https://undeadly.org/cgi?action=article;sid=20251230094932

FreeBSD 15.0 – Composable by Design.
https://youtube.com/watch?v=gvJc5qHVLzg

Thank You for Amazing Year – Happy New Year from Gary.
https://youtube.com/watch?v=KZgZaBxrAG0

Updates to pf(4) Packet Filter in FreeBSD and pfSense Software.
https://netgate.com/blog/updates-to-the-pf-packet-filter-in-freebsd-and-pfsense-software

FreeBSD Driver for ws2812 RGP LED String.
http://ketas.si.pri.ee/misc/ws2812/ws2812.html

FreeBSD Net Autotuner – Dynamically Adjust TCP Stack/Congestion Control Algorithm/Buffers/Queues.
https://github.com/geobarrod/FBSD-Net-Autotuner

Send Your HDD and Get ZFS VPS on It.
https://zfs.rent/

Managing FreeBSD Jails with Ansible with jailexec Connection Plugin.
https://blog.hofstede.it/managing-freebsd-jails-with-ansible-the-jailexec-connection-plugin/

Devuan 6.1 Released Init Freedom without systemd.
https://phoronix.com/news/Devuan-6.1-Released

IceWM 4.0 Improves [ALT]-[TAB] Window Switcher/Alpha Blending/32bit RGBA Default.
https://phoronix.com/news/IceWM-4.0-Released

Sentinel Immune – Biological Immunity for IT Security on DragonflyBSD.
https://dev.to/dmitry_labintcev_9e611e04/sentinel-immune-biological-immunity-for-it-security-279f
https://github.com/DmitrL-dev/AISecurity

Samba Jail Made with FreeBSD AppJail.
https://github.com/ab-ten/smbjail

WebZFS is Modern Web Based Management Interface for ZFS Pools/Datasets/Snapshots/S.M.A.R.T Monitoring.
https://github.com/webzfs/webzfs

AMIGA Desktops Worth Seeing.
https://datagubbe.se/wbshots/

AMIGA 3000 UNIX and Sun Microsystems.
https://datagubbe.se/amix/

FreeBSD Runs Perfectly on 16 Years Old Laptop.
https://lemmy.sdf.org/post/48417779

FreeBSD Upgrade to 15.
https://euroquis.nl/freebsd/2026/01/02/freebsd.html

FreeBSD 14 to 15 Upgrade (Intel).
https://euroquis.nl/freebsd/2026/01/03/freebsd.html

BSD Weekly – Issue 260.
https://bsdweekly.com/issues/260

BSD Now 644 – Holidays 2025 – What You Neen Doin?
https://www.bsdnow.tv/644

On Building Ansible Training Environment on FreeBSD. [2019]
https://jpmens.net/2019/12/11/on-building-an-ansible-training-environment-on-freebsd/

Fixing Buffer Overflow in UNIX V4 Like It is 1973.
https://sigma-star.at/blog/2025/12/unix-v4-buffer-overflow/

How to Install FreeBSD 15.0.
https://youtube.com/watch?v=JSTFpe70tOo

Can I Finally Start Using Wayland in 2026?
https://michael.stapelberg.ch/posts/2026-01-04-wayland-sway-in-2026/

GhostBSD – Step by Step Install.
https://youtube.com/watch?v=gjw2OykX5_o

Hardware

New in Collection – Part 5 – New Hermes NHI 810.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-5-new-hermes-nhi-810/

Cameras and Lenses – Bartosz Ciechanowski.
https://ciechanow.ski/cameras-and-lenses/

Samsung SDI Batteries with 600 Mile Range That Charge in 9 Minutes Ready for Production.
https://goodnewsnetwork.org/samsungs-600-mile-range-batteries-that-charge-in-9-minutes-ready-for-production-sale-next-year/

Why One Man is Fighting for Our Right to Control Our Garage Door Openers.
https://nytimes.com/2025/12/04/technology/personaltech/why-one-man-is-fighting-for-our-right-to-control-our-garage-door-openers.html

2026 Dodge Durango Drops V6 and Goes V8 Only Across Entire Lineup.
https://caranddriver.com/news/a65656950/2026-dodge-durango-v8-engine-only/

Life

Between Then and Now.
https://my-notes.dragas.net/2025/12/29/between-then-and-now/

2025 State of Men UK Report.
https://beyondequality.org/blog-posts/state-of-men-uk-report

Other

FFmpeg Developer Files DMCA Against Rockchip After 2 Year Wait for License Fix.
https://it.slashdot.org/story/25/12/26/193244/ffmpeg-developer-files-dmca-against-rockchip-after-two-year-wait-for-license-fix

Web Browsers Have Stopped Blocking Pop Ups.
https://smokingonabike.com/2025/12/31/web-browsers-have-stopped-blocking-pop-ups/

I Tried Servo – Undercover Web Browser Engine Made with Rust.
https://spacebar.news/servo-undercover-web-browser-engine/

GOG is Getting Acquired by Its Original Cofounder.
https://gog.com/blog/gog-is-getting-acquired-by-its-original-co-founder-what-it-means-for-you/

Heroes ]I[ – Horn of the Abyss – New Bulwark City.
https://youtube.com/watch?v=lfbJF3YHRgo

Windows XP/Vista/7/8.1/10/11 Speed Test.
https://youtube.com/watch?v=7VZJO-hOT4c&t=862s

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

The Kimwolf Botnet is Stalking Your Local Network

Post by Brian Krebs via Krebs on Security »

The story you are reading is a series of scoops nestled inside a far more urgent Internet-wide security advisory. The vulnerability at issue has been exploited for months already, and it’s time for a broader awareness of the threat. The short version is that everything you thought you knew about the security of the internal network behind your Internet router probably is now dangerously out of date.

The security company Synthient currently sees more than 2 million infected Kimwolf devices distributed globally but with concentrations in Vietnam, Brazil, India, Saudi Arabia, Russia and the United States. Synthient found that two-thirds of the Kimwolf infections are Android TV boxes with no security or authentication built in.

The past few months have witnessed the explosive growth of a new botnet dubbed Kimwolf, which experts say has infected more than 2 million devices globally. The Kimwolf malware forces compromised systems to relay malicious and abusive Internet traffic — such as ad fraud, account takeover attempts and mass content scraping — and participate in crippling distributed denial-of-service (DDoS) attacks capable of knocking nearly any website offline for days at a time.

More important than Kimwolf’s staggering size, however, is the diabolical method it uses to spread so quickly: By effectively tunneling back through various “residential proxy” networks and into the local networks of the proxy endpoints, and by further infecting devices that are hidden behind the assumed protection of the user’s firewall and Internet router.

Residential proxy networks are sold as a way for customers to anonymize and localize their Web traffic to a specific region, and the biggest of these services allow customers to route their traffic through devices in virtually any country or city around the globe.

The malware that turns an end-user’s Internet connection into a proxy node is often bundled with dodgy mobile apps and games. These residential proxy programs also are commonly installed via unofficial Android TV boxes sold by third-party merchants on popular e-commerce sites like Amazon, BestBuy, Newegg, and Walmart.

These TV boxes range in price from $40 to $400, are marketed under a dizzying range of no-name brands and model numbers, and frequently are advertised as a way to stream certain types of subscription video content for free. But there’s a hidden cost to this transaction: As we’ll explore in a moment, these TV boxes make up a considerable chunk of the estimated two million systems currently infected with Kimwolf.

Some of the unsanctioned Android TV boxes that come with residential proxy malware pre-installed. Image: Synthient.

Kimwolf also is quite good at infecting a range of Internet-connected digital photo frames that likewise are abundant at major e-commerce websites. In November 2025, researchers from Quokka published a report (PDF) detailing serious security issues in Android-based digital picture frames running the Uhale app — including Amazon’s bestselling digital frame as of March 2025.

There are two major security problems with these photo frames and unofficial Android TV boxes. The first is that a considerable percentage of them come with malware pre-installed, or else require the user to download an unofficial Android App Store and malware in order to use the device for its stated purpose (video content piracy). The most typical of these uninvited guests are small programs that turn the device into a residential proxy node that is resold to others.

The second big security nightmare with these photo frames and unsanctioned Android TV boxes is that they rely on a handful of Internet-connected microcomputer boards that have no discernible security or authentication requirements built-in. In other words, if you are on the same network as one or more of these devices, you can likely compromise them simultaneously by issuing a single command across the network.

THERE’S NO PLACE LIKE 127.0.0.1

The combination of these two security realities came to the fore in October 2025, when an undergraduate computer science student at the Rochester Institute of Technology began closely tracking Kimwolf’s growth, and interacting directly with its apparent creators on a daily basis.

Benjamin Brundage is the 22-year-old founder of the security firm Synthient, a startup that helps companies detect proxy networks and learn how those networks are being abused. Conducting much of his research into Kimwolf while studying for final exams, Brundage told KrebsOnSecurity in late October 2025 he suspected Kimwolf was a new Android-based variant of Aisuru, a botnet that was incorrectly blamed for a number of record-smashing DDoS attacks last fall.

Brundage says Kimwolf grew rapidly by abusing a glaring vulnerability in many of the world’s largest residential proxy services. The crux of the weakness, he explained, was that these proxy services weren’t doing enough to prevent their customers from forwarding requests to internal servers of the individual proxy endpoints.

Most proxy services take basic steps to prevent their paying customers from “going upstream” into the local network of proxy endpoints, by explicitly denying requests for local addresses specified in RFC-1918, including the well-known Network Address Translation (NAT) ranges 10.0.0.0/8, 192.168.0.0/16, and 172.16.0.0/12. These ranges allow multiple devices in a private network to access the Internet using a single public IP address, and if you run any kind of home or office network, your internal address space operates within one or more of these NAT ranges.

However, Brundage discovered that the people operating Kimwolf had figured out how to talk directly to devices on the internal networks of millions of residential proxy endpoints, simply by changing their Domain Name System (DNS) settings to match those in the RFC-1918 address ranges.

“It is possible to circumvent existing domain restrictions by using DNS records that point to 192.168.0.1 or 0.0.0.0,” Brundage wrote in a first-of-its-kind security advisory sent to nearly a dozen residential proxy providers in mid-December 2025. “This grants an attacker the ability to send carefully crafted requests to the current device or a device on the local network. This is actively being exploited, with attackers leveraging this functionality to drop malware.”

As with the digital photo frames mentioned above, many of these residential proxy services run solely on mobile devices that are running some game, VPN or other app with a hidden component that turns the user’s mobile phone into a residential proxy — often without any meaningful consent.

In a report published today, Synthient said key actors involved in Kimwolf were observed monetizing the botnet through app installs, selling residential proxy bandwidth, and selling its DDoS functionality.

“Synthient expects to observe a growing interest among threat actors in gaining unrestricted access to proxy networks to infect devices, obtain network access, or access sensitive information,” the report observed. “Kimwolf highlights the risks posed by unsecured proxy networks and their viability as an attack vector.”

ANDROID DEBUG BRIDGE

After purchasing a number of unofficial Android TV box models that were most heavily represented in the Kimwolf botnet, Brundage further discovered the proxy service vulnerability was only part of the reason for Kimwolf’s rapid rise: He also found virtually all of the devices he tested were shipped from the factory with a powerful feature called Android Debug Bridge (ADB) mode enabled by default.

Many of the unofficial Android TV boxes infected by Kimwolf include the ominous disclaimer: “Made in China. Overseas use only.” Image: Synthient.

ADB is a diagnostic tool intended for use solely during the manufacturing and testing processes, because it allows the devices to be remotely configured and even updated with new (and potentially malicious) firmware. However, shipping these devices with ADB turned on creates a security nightmare because in this state they constantly listen for and accept unauthenticated connection requests.

For example, opening a command prompt and typing “adb connect” along with a vulnerable device’s (local) IP address followed immediately by “:5555” will very quickly offer unrestricted “super user” administrative access.

Brundage said by early December, he’d identified a one-to-one overlap between new Kimwolf infections and proxy IP addresses offered for rent by China-based IPIDEA, currently the world’s largest residential proxy network by all accounts.

“Kimwolf has almost doubled in size this past week, just by exploiting IPIDEA’s proxy pool,” Brundage told KrebsOnSecurity in early December as he was preparing to notify IPIDEA and 10 other proxy providers about his research.

Brundage said Synthient first confirmed on December 1, 2025 that the Kimwolf botnet operators were tunneling back through IPIDEA’s proxy network and into the local networks of systems running IPIDEA’s proxy software. The attackers dropped the malware payload by directing infected systems to visit a specific Internet address and to call out the pass phrase “krebsfiveheadindustries” in order to unlock the malicious download.

On December 30, Synthient said it was tracking roughly 2 million IPIDEA addresses exploited by Kimwolf in the previous week. Brundage said he has witnessed Kimwolf rebuilding itself after one recent takedown effort targeting its control servers — from almost nothing to two million infected systems just by tunneling through proxy endpoints on IPIDEA for a couple of days.

Brundage said IPIDEA has a seemingly inexhaustible supply of new proxies, advertising access to more than 100 million residential proxy endpoints around the globe in the past week alone. Analyzing the exposed devices that were part of IPIDEA’s proxy pool, Synthient said it found more than two-thirds were Android devices that could be compromised with no authentication needed.

SECURITY NOTIFICATION AND RESPONSE

After charting a tight overlap in Kimwolf-infected IP addresses and those sold by IPIDEA, Brundage was eager to make his findings public: The vulnerability had clearly been exploited for several months, although it appeared that only a handful of cybercrime actors were aware of the capability. But he also knew that going public without giving vulnerable proxy providers an opportunity to understand and patch it would only lead to more mass abuse of these services by additional cybercriminal groups.

On December 17, Brundage sent a security notification to all 11 of the apparently affected proxy providers, hoping to give each at least a few weeks to acknowledge and address the core problems identified in his report before he went public. Many proxy providers who received the notification were resellers of IPIDEA that white-labeled the company’s service.

KrebsOnSecurity first sought comment from IPIDEA in October 2025, in reporting on a story about how the proxy network appeared to have benefitted from the rise of the Aisuru botnet, whose administrators appeared to shift from using the botnet primarily for DDoS attacks to simply installing IPIDEA’s proxy program, among others.

On December 25, KrebsOnSecurity received an email from an IPIDEA employee identified only as “Oliver,” who said allegations that IPIDEA had benefitted from Aisuru’s rise were baseless.

“After comprehensively verifying IP traceability records and supplier cooperation agreements, we found no association between any of our IP resources and the Aisuru botnet, nor have we received any notifications from authoritative institutions regarding our IPs being involved in malicious activities,” Oliver wrote. “In addition, for external cooperation, we implement a three-level review mechanism for suppliers, covering qualification verification, resource legality authentication and continuous dynamic monitoring, to ensure no compliance risks throughout the entire cooperation process.”

“IPIDEA firmly opposes all forms of unfair competition and malicious smearing in the industry, always participates in market competition with compliant operation and honest cooperation, and also calls on the entire industry to jointly abandon irregular and unethical behaviors and build a clean and fair market ecosystem,” Oliver continued.

Meanwhile, the same day that Oliver’s email arrived, Brundage shared a response he’d just received from IPIDEA’s security officer, who identified himself only by the first name Byron. The security officer said IPIDEA had made a number of important security changes to its residential proxy service to address the vulnerability identified in Brundage’s report.

“By design, the proxy service does not allow access to any internal or local address space,” Byron explained. “This issue was traced to a legacy module used solely for testing and debugging purposes, which did not fully inherit the internal network access restrictions. Under specific conditions, this module could be abused to reach internal resources. The affected paths have now been fully blocked and the module has been taken offline.”

Byron told Brundage IPIDEA also instituted multiple mitigations for blocking DNS resolution to internal (NAT) IP ranges, and that it was now blocking proxy endpoints from forwarding traffic on “high-risk” ports “to prevent abuse of the service for scanning, lateral movement, or access to internal services.”

An excerpt from an email sent by IPIDEA’s security officer in response to Brundage’s vulnerability notification. Click to enlarge.

Brundage said IPIDEA appears to have successfully patched the vulnerabilities he identified. He also noted he never observed the Kimwolf actors targeting proxy services other than IPIDEA, which has not responded to requests for comment.

Riley Kilmer is founder of Spur.us, a technology firm that helps companies identify and filter out proxy traffic. Kilmer said Spur has tested Brundage’s findings and confirmed that IPIDEA and all of its affiliate resellers indeed allowed full and unfiltered access to the local LAN.

Kilmer said one model of unsanctioned Android TV boxes that is especially popular — the Superbox, which we profiled in November’s Is Your Android TV Streaming Box Part of a Botnet? — leaves Android Debug Mode running on localhost:5555.

“And since Superbox turns the IP into an IPIDEA proxy, a bad actor just has to use the proxy to localhost on that port and install whatever bad SDKs [software development kits] they want,” Kilmer told KrebsOnSecurity.

Superbox media streaming boxes for sale on Walmart.com.

ECHOES FROM THE PAST

Both Brundage and Kilmer say IPIDEA appears to be the second or third reincarnation of a residential proxy network formerly known as 911S5 Proxy, a service that operated between 2014 and 2022 and was wildly popular on cybercrime forums. 911S5 Proxy imploded a week after KrebsOnSecurity published a deep dive on the service’s sketchy origins and leadership in China.

In that 2022 profile, we cited work by researchers at the University of Sherbrooke in Canada who were studying the threat 911S5 could pose to internal corporate networks. The researchers noted that “the infection of a node enables the 911S5 user to access shared resources on the network such as local intranet portals or other services.”

“It also enables the end user to probe the LAN network of the infected node,” the researchers explained. “Using the internal router, it would be possible to poison the DNS cache of the LAN router of the infected node, enabling further attacks.”

911S5 initially responded to our reporting in 2022 by claiming it was conducting a top-down security review of the service. But the proxy service abruptly closed up shop just one week later, saying a malicious hacker had destroyed all of the company’s customer and payment records. In July 2024, The U.S. Department of the Treasury sanctioned the alleged creators of 911S5, and the U.S. Department of Justice arrested the Chinese national named in my 2022 profile of the proxy service.

Kilmer said IPIDEA also operates a sister service called 922 Proxy, which the company has pitched from Day One as a seamless alternative to 911S5 Proxy.

“You cannot tell me they don’t want the 911 customers by calling it that,” Kilmer said.

Among the recipients of Synthient’s notification was the proxy giant Oxylabs. Brundage shared an email he received from Oxylabs’ security team on December 31, which acknowledged Oxylabs had started rolling out security modifications to address the vulnerabilities described in Synthient’s report.

Reached for comment, Oxylabs confirmed they “have implemented changes that now eliminate the ability to bypass the blocklist and forward requests to private network addresses using a controlled domain.” But it said there is no evidence that Kimwolf or other other attackers exploited its network.

“In parallel, we reviewed the domains identified in the reported exploitation activity and did not observe traffic associated with them,” the Oxylabs statement continued. “Based on this review, there is no indication that our residential network was impacted by these activities.”

PRACTICAL IMPLICATIONS

Consider the following scenario, in which the mere act of allowing someone to use your Wi-Fi network could lead to a Kimwolf botnet infection. In this example, a friend or family member comes to stay with you for a few days, and you grant them access to your Wi-Fi without knowing that their mobile phone is infected with an app that turns the device into a residential proxy node. At that point, your home’s public IP address will show up for rent at the website of some residential proxy provider.

Miscreants like those behind Kimwolf then use residential proxy services online to access that proxy node on your IP, tunnel back through it and into your local area network (LAN), and automatically scan the internal network for devices with Android Debug Bridge mode turned on.

By the time your guest has packed up their things, said their goodbyes and disconnected from your Wi-Fi, you now have two devices on your local network — a digital photo frame and an unsanctioned Android TV box — that are infected with Kimwolf. You may have never intended for these devices to be exposed to the larger Internet, and yet there you are.

Here’s another possible nightmare scenario: Attackers use their access to proxy networks to modify your Internet router’s settings so that it relies on malicious DNS servers controlled by the attackers — allowing them to control where your Web browser goes when it requests a website. Think that’s far-fetched? Recall the DNSChanger malware from 2012 that infected more than a half-million routers with search-hijacking malware, and ultimately spawned an entire security industry working group focused on containing and eradicating it.

XLAB

Much of what is published so far on Kimwolf has come from the Chinese security firm XLab, which was the first to chronicle the rise of the Aisuru botnet in late 2024. In its latest blog post, XLab said it began tracking Kimwolf on October 24, when the botnet’s control servers were swamping Cloudflare’s DNS servers with lookups for the distinctive domain 14emeliaterracewestroxburyma02132[.]su.

This domain and others connected to early Kimwolf variants spent several weeks topping Cloudflare’s chart of the Internet’s most sought-after domains, edging out Google.com and Apple.com of their rightful spots in the top 5 most-requested domains. That’s because during that time Kimwolf was asking its millions of bots to check in frequently using Cloudflare’s DNS servers.

The Chinese security firm XLab found the Kimwolf botnet had enslaved between 1.8 and 2 million devices, with heavy concentrations in Brazil, India, The United States of America and Argentina. Image: blog.xLab.qianxin.com

It is clear from reading the XLab report that KrebsOnSecurity (and security experts) probably erred in misattributing some of Kimwolf’s early activities to the Aisuru botnet, which appears to be operated by a different group entirely. IPDEA may have been truthful when it said it had no affiliation with the Aisuru botnet, but Brundage’s data left no doubt that its proxy service clearly was being massively abused by Aisuru’s Android variant, Kimwolf.

XLab said Kimwolf has infected at least 1.8 million devices, and has shown it is able to rebuild itself quickly from scratch.

“Analysis indicates that Kimwolf’s primary infection targets are TV boxes deployed in residential network environments,” XLab researchers wrote. “Since residential networks usually adopt dynamic IP allocation mechanisms, the public IPs of devices change over time, so the true scale of infected devices cannot be accurately measured solely by the quantity of IPs. In other words, the cumulative observation of 2.7 million IP addresses does not equate to 2.7 million infected devices.”

XLab said measuring Kimwolf’s size also is difficult because infected devices are distributed across multiple global time zones. “Affected by time zone differences and usage habits (e.g., turning off devices at night, not using TV boxes during holidays, etc.), these devices are not online simultaneously, further increasing the difficulty of comprehensive observation through a single time window,” the blog post observed.

XLab noted that the Kimwolf author shows an almost ‘obsessive’ fixation” on Yours Truly, apparently leaving “easter eggs” related to my name in multiple places through the botnet’s code and communications:

Image: XLAB.

ANALYSIS AND ADVICE

One frustrating aspect of threats like Kimwolf is that in most cases it is not easy for the average user to determine if there are any devices on their internal network which may be vulnerable to threats like Kimwolf and/or already infected with residential proxy malware.

Let’s assume that through years of security training or some dark magic you can successfully identify that residential proxy activity on your internal network was linked to a specific mobile device inside your house: From there, you’d still need to isolate and remove the app or unwanted component that is turning the device into a residential proxy.

Also, the tooling and knowledge needed to achieve this kind of visibility just isn’t there from an average consumer standpoint. The work that it takes to configure your network so you can see and interpret logs of all traffic coming in and out is largely beyond the skillset of most Internet users (and, I’d wager, many security experts). But it’s a topic worth exploring in an upcoming story.

Happily, Synthient has erected a page on its website that will state whether a visitor’s public Internet address was seen among those of Kimwolf-infected systems. Brundage also has compiled a list of the unofficial Android TV boxes that are most highly represented in the Kimwolf botnet.

If you own a TV box that matches one of these model names and/or numbers, please just rip it out of your network. If you encounter one of these devices on the network of a family member or friend, send them a link to this story and explain that it’s not worth the potential hassle and harm created by keeping them plugged in.

The top 15 product devices represented in the Kimwolf botnet, according to Synthient.

Chad Seaman is a principal security researcher with Akamai Technologies. Seaman said he wants more consumers to be wary of these pseudo Android TV boxes to the point where they avoid them altogether.

“I want the consumer to be paranoid of these crappy devices and of these residential proxy schemes,” he said. “We need to highlight why they’re dangerous to everyone and to the individual. The whole security model where people think their LAN (Local Internal Network) is safe, that there aren’t any bad guys on the LAN so it can’t be that dangerous is just really outdated now.”

“The idea that an app can enable this type of abuse on my network and other networks, that should really give you pause,” about which devices to allow onto your local network, Seaman said. “And it’s not just Android devices here. Some of these proxy services have SDKs for Mac and Windows, and the iPhone. It could be running something that inadvertently cracks open your network and lets countless random people inside.”

In July 2025, Google filed a “John Doe” lawsuit (PDF) against 25 unidentified defendants collectively dubbed the “BadBox 2.0 Enterprise,” which Google described as a botnet of over ten million unsanctioned Android streaming devices engaged in advertising fraud. Google said the BADBOX 2.0 botnet, in addition to compromising multiple types of devices prior to purchase, also can infect devices by requiring the download of malicious apps from unofficial marketplaces.

Google’s lawsuit came on the heels of a June 2025 advisory from the Federal Bureau of Investigation (FBI), which warned that cyber criminals were gaining unauthorized access to home networks by either configuring the products with malware prior to the user’s purchase, or infecting the device as it downloads required applications that contain backdoors — usually during the set-up process.

The FBI said BADBOX 2.0 was discovered after the original BADBOX campaign was disrupted in 2024. The original BADBOX was identified in 2023, and primarily consisted of Android operating system devices that were compromised with backdoor malware prior to purchase.

Lindsay Kaye is vice president of threat intelligence at HUMAN Security, a company that worked closely on the BADBOX investigations. Kaye said the BADBOX botnets and the residential proxy networks that rode on top of compromised devices were detected because they enabled a ridiculous amount of advertising fraud, as well as ticket scalping, retail fraud, account takeovers and content scraping.

Kaye said consumers should stick to known brands when it comes to purchasing things that require a wired or wireless connection.

“If people are asking what they can do to avoid being victimized by proxies, it’s safest to stick with name brands,” Kaye said. “Anything promising something for free or low-cost, or giving you something for nothing just isn’t worth it. And be careful about what apps you allow on your phone.”

Many wireless routers these days make it relatively easy to deploy a “Guest” wireless network on-the-fly. Doing so allows your guests to browse the Internet just fine but it blocks their device from being able to talk to other devices on the local network — such as shared folders, printers and drives. If someone — a friend, family member, or contractor — requests access to your network, give them the guest Wi-Fi network credentials if you have that option.

There is a small but vocal pro-piracy camp that is almost condescendingly dismissive of the security threats posed by these unsanctioned Android TV boxes. These tech purists positively chafe at the idea of people wholesale discarding one of these TV boxes. A common refrain from this camp is that Internet-connected devices are not inherently bad or good, and that even factory-infected boxes can be flashed with new firmware or custom ROMs that contain no known dodgy software.

However, it’s important to point out that the majority of people buying these devices are not security or hardware experts; the devices are sought out because they dangle something of value for “free.” Most buyers have no idea of the bargain they’re making when plugging one of these dodgy TV boxes into their network.

It is somewhat remarkable that we haven’t yet seen the entertainment industry applying more visible pressure on the major e-commerce vendors to stop peddling this insecure and actively malicious hardware that is largely made and marketed for video piracy. These TV boxes are a public nuisance for bundling malicious software while having no apparent security or authentication built-in, and these two qualities make them an attractive nuisance for cybercriminals.

Stay tuned for Part II in this series, which will poke through clues left behind by the people who appear to have built Kimwolf and benefited from it the most.

Top

Linux kernel security work

Post by Greg Kroah-Hartman via Linux Kernel Monkey Log »

Lots of the CVE world seems to focus on “security bugs” but I’ve found that it is not all that well known exactly how the Linux kernel security process works. I gave a talk about this back in 2023 and at other conferences since then, attempting to explain how it works, but I also thought it would be good to explain this all in writing as it is required to know this when trying to understand how the Linux kernel CNA issues CVEs.

Top

Fwd: The "60 Minutes" segment about the CECOT prison that was pulled last minute

Post by Sebastian Pipping via Hartwork Blog »

When I tried watching The 60 Minutes Story The Trump Administration Doesn't Want You To See from my bookmarks today, I got error…

Video unavailable

This video is no longer available due to a copyright claim by Paramount Global companies[.]

…and so below you can find a re-upload if you also are curious what the pulled video is about:

Original title: The 60 Minutes story the Trump regime did not want you to see

For additional context:

Original title: Chris Murphy: Trump Has Taken 'Editorial Control Of CBS' After 60 Minutes Pulls Critical Segment

Top

Happy 16th Birthday, KrebsOnSecurity.com!

Post by Brian Krebs via Krebs on Security »

KrebsOnSecurity.com celebrates its 16th anniversary today! A huge “thank you” to all of our readers — newcomers, long-timers and drive-by critics alike. Your engagement this past year here has been tremendous and truly a salve on a handful of dark days. Happily, comeuppance was a strong theme running through our coverage in 2025, with a primary focus on entities that enabled complex and globally-dispersed cybercrime services.

Image: Shutterstock, Younes Stiller Kraske.

In May 2024, we scrutinized the history and ownership of Stark Industries Solutions Ltd., a “bulletproof hosting” provider that came online just two weeks before Russia invaded Ukraine and served as a primary staging ground for repeated Kremlin cyberattacks and disinformation efforts. A year later, Stark and its two co-owners were sanctioned by the European Union, but our analysis showed those penalties have done little to stop the Stark proprietors from rebranding and transferring considerable network assets to other entities they control.

In December 2024, KrebsOnSecurity profiled Cryptomus, a financial firm registered in Canada that emerged as the payment processor of choice for dozens of Russian cryptocurrency exchanges and websites hawking cybercrime services aimed at Russian-speaking customers. In October 2025, Canadian financial regulators ruled that Cryptomus had grossly violated its anti-money laundering laws, and levied a record $176 million fine against the platform.

In September 2023, KrebsOnSecurity published findings from researchers who concluded that a series of six-figure cyberheists across dozens of victims resulted from thieves cracking master passwords stolen from the password manager service LastPass in 2022. In a court filing in March 2025, U.S. federal agents investigating a spectacular $150 million cryptocurrency heist said they had reached the same conclusion.

Phishing was a major theme of this year’s coverage, which peered inside the day-to-day operations of several voice phishing gangs that routinely carried out elaborate, convincing, and financially devastating cryptocurrency thefts. A Day in the Life of a Prolific Voice Phishing Crew examined how one cybercrime gang abused legitimate services at Apple and Google to force a variety of outbound communications to their users, including emails, automated phone calls and system-level messages sent to all signed-in devices.

Nearly a half-dozen stories in 2025 dissected the incessant SMS phishing or “smishing” coming from China-based phishing kit vendors, who make it easy for customers to convert phished payment card data into mobile wallets from Apple and Google. In an effort to wrest control over this phishing syndicate’s online resources, Google has since filed at least two John Doe lawsuits targeting these groups and dozens of unnamed defendants.

In January, we highlighted research into a dodgy and sprawling content delivery network called Funnull that specialized in helping China-based gambling and money laundering websites distribute their operations across multiple U.S.-based cloud providers. Five months later, the U.S. government sanctioned Funnull, identifying it as a top source of investment/romance scams known as “pig butchering.”

Image: Shutterstock, ArtHead.

In May, Pakistan arrested 21 people alleged to be working for Heartsender, a phishing and malware dissemination service that KrebsOnSecurity first profiled back in 2015. The arrests came shortly after the FBI and the Dutch police seized dozens of servers and domains for the group. Many of those arrested were first publicly identified in a 2021 story here about how they’d inadvertently infected their computers with malware that gave away their real-life identities.

In April, the U.S. Department of Justice indicted the proprietors of a Pakistan-based e-commerce company for conspiring to distribute synthetic opioids in the United States. The following month, KrebsOnSecurity detailed how the proprietors of the sanctioned entity are perhaps better known for operating an elaborate and lengthy scheme to scam westerners seeking help with trademarks, book writing, mobile app development and logo designs.

Earlier this month, we examined an academic cheating empire turbocharged by Google Ads that earned tens of millions of dollars in revenue and has curious ties to a Kremlin-connected oligarch whose Russian university builds drones for Russia’s war against Ukraine.

An attack drone advertised on a website hosted in the same network as Russia’s largest private education company — Synergy University.

As ever, KrebsOnSecurity endeavored to keep close tabs on the world’s biggest and most disruptive botnets, which pummeled the Internet this year with distributed denial-of-service (DDoS) assaults that were two to three times the size and impact of previous record DDoS attacks.

In June, KrebsOnSecurity.com was hit by the largest DDoS attack that Google had ever mitigated at the time (we are a grateful guest of Google’s excellent Project Shield offering). Experts blamed that attack on an Internet-of-Things botnet called Aisuru that had rapidly grown in size and firepower since its debut in late 2024. Another Aisuru attack on Cloudflare just days later practically doubled the size of the June attack against this website. Not long after that, Aisuru was blamed for a DDoS that again doubled the previous record.

In October, it appeared the cybercriminals in control of Aisuru had shifted the botnet’s focus from DDoS to a more sustainable and profitable use: Renting hundreds of thousands of infected Internet of Things (IoT) devices to proxy services that help cybercriminals anonymize their traffic.

However, it has recently become clear that at least some of the disruptive botnet and residential proxy activity attributed to Aisuru last year likely was the work of people responsible for building and testing a powerful botnet known as Kimwolf. Chinese security firm XLab, which was the first to chronicle Aisuru’s rise in 2024, recently profiled Kimwolf as easily the world’s biggest and most dangerous collection of compromised machines — with approximately 1.83 million devices under its thumb as of December 17.

XLab noted that the Kimwolf author “shows an almost ‘obsessive’ fixation on the well-known cybersecurity investigative journalist Brian Krebs, leaving easter eggs related to him in multiple places.”

Image: XLab, Kimwolf Botnet Exposed: The Massive Android Botnet with 1.8 million infected devices.

I am happy to report that the first KrebsOnSecurity stories of 2026 will go deep into the origins of Kimwolf, and examine the botnet’s unique and highly invasive means of spreading digital disease far and wide. The first in that series will include a somewhat sobering and global security notification concerning the devices and residential proxy services that are inadvertently helping to power Kimwolf’s rapid growth.

Thank you once again for your continued readership, encouragement and support. If you like the content we publish at KrebsOnSecurity.com, please consider making an exception for our domain in your ad blocker. The ads we run are limited to a handful of static images that are all served in-house and vetted by me (there is no third-party content on this site, period). Doing so would help further support the work you see here almost every week.

And if you haven’t done so yet, sign up for our email newsletter! (62,000 other subscribers can’t be wrong, right?). The newsletter is just a plain text email that goes out the moment a new story is published. We send between one and two emails a week, we never share our email list, and we don’t run surveys or promotions.

Thanks again, and Happy New Year everyone! Be safe out there.

Top

Valuable News – 2025/12/29

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD 15 Version of PF Has Basically Caught Up to OpenBSD.
https://utcc.utoronto.ca/~cks/space/blog/unix/FreeBSDPFHasCaughtUp

Updates to pf(4) Packet Filter in FreeBSD and pfSense Software.
https://netgate.com/blog/updates-to-the-pf-packet-filter-in-freebsd-and-pfsense-software

XLibre 25.1 – Future of X11 Adds New Platforms and New Features.
https://x.com/LundukeJournal/status/2003182372100145594

Lets Run FreeBSD 15.0-RELEASE on Raspberry Pi Zero 2W.
https://briancallahan.net/blog/20251216.html

Wireguard FreeBSD Way.
https://blog.feld.me/posts/2025/12/wireguard-freebsd-way/

BSD Now 642: Look Harder.
https://www.bsdnow.tv/642

Moving from Windows to FreeBSD as Linux Chaos Alternative.
https://hackaday.com/2025/11/11/moving-from-windows-to-freebsd-as-the-linux-chaos-alternative/

FreeBSD Foundation 2025 Software Development and Infrastructure Support.
https://freebsdfoundation.org/blog/2025-software-development-and-infrastructure-support/

The pkgsrc 2025Q4 is Released.
https://bsdsec.net/articles/pkgsrc-2025q4-released

Lets Setup monitord on FreeBSD.
https://youtube.com/watch?v=WMX9u7We3ys

Initial Analysis of Discovered UNIX V4 Tape.
https://spinellis.gr/blog/20251223/

UNIX V4 Tape Successfully Recovered.
https://theregister.com/2025/12/23/unix_v4_tape_successfully_recovered/

Finally Upgraded Fleet to FreeBSD 15.
https://rubenerd.com/finally-upgraded-to-freebsd-15/

FreeBSD Home NAS – Part 1 – Configuring ZFS Mirror (RAID1).
https://rtfm.co.ua/en/freebsd-home-nas-part-1-configuring-zfs-mirror-raid1/

FreeBSD Home NAS – Part 2 – Introduction to Packet Filter (PF) Firewall.
https://rtfm.co.ua/en/freebsd-home-nas-part-2-introduction-to-packet-filter-pf-firewall/

FreeBSD Home NAS – Part 3 – WireGuard VPN/Linux Peer/Routing.
https://rtfm.co.ua/en/freebsd-home-nas-part-3-wireguard-vpn-linux-peer-and-routing/

FreeBSD Home NAS – Part 4 – Local DNS with Unbound.
https://rtfm.co.ua/en/freebsd-home-nas-part-4-local-dns-with-unbound/

FreeBSD Home NAS – Part 5 – ZFS Pool/Datasets/Snapshots/Monitoring.
https://rtfm.co.ua/en/freebsd-home-nas-part-5-zfs-pool-datasets-snapshots-and-zfs-monitoring/

FreeBSD Home NAS – Part 6 – Samba Server and Client Connections.
https://rtfm.co.ua/en/freebsd-home-nas-part-6-samba-server-and-client-connections/

XLibre XServer 25.1 Changes.
https://github.com/X11Libre/xserver/wiki/XLibre-XServer-25.1-Changes

Streaming HiFi Audio from FreeBSD NAS to OpenBSD Playback Machine Using MPD and sndio.
https://vincentdelft.be/post/post_20251225

Phoenix is New X11 Server Written from Scratch in Zig.
https://git.dec05eba.com/phoenix/about/

OpenBSD Games Database.
https://github.com/playonbsd/OpenBSD-Games-Database

The n8n on NetBSD.
https://drsaracco.wordpress.com/2025/12/25/n8n-netbsd/

How to Automatically Start or Reattach to tmux(1) Session with SSH.
https://rkta.de/ssh-tmux-reattach.html

Linux Foundation Makes $300 Million in 2025 and Spends 2.6% on Linux.
https://rumble.com/v73dyp6-linux-foundation-makes-300-million-in-2025-spends-2.6-on-linux-says-ai-is-t.html

FreeBSD Git Weekly: 2025-12-15 to 2025-12-21.
https://freebsd-git-weekly.tarsnap.net/2025-12-15.html

FreeBSD Laptop and Desktop Workgroup – Call 13.
https://youtube.com/watch?v=NNKFHEZw3eA

The ssh-agent(1) Broken in tmux(1) Resolved.
https://blogsystem5.substack.com/p/ssh-agent-switcher-release

Critical n8n Flaw (CVSS 9.9) Enables Arbitrary Code Execution Across Thousands of Instances.
https://thehackernews.com/2025/12/critical-n8n-flaw-cvss-99-enables.html

My Journey into FreeBSD and Glorious Manual – Love Story.
https://andreiclinciu.net/freebsd-experience/

BSD Now 643: Unwrapping Gifts.
https://www.bsdnow.tv/643

All Aboard 15.0-RELEASE Train!
https://vulcanridr.mataroa.blog/blog/all-aboard-the-150-release-train/

Basic nc(1) Commands.
https://drsaracco.wordpress.com/2013/06/20/basic-netcat-commands/

Fast fastcompmgr(1) Compositor for X11.
https://github.com/tycho-kirchner/fastcompmgr

XLibre Xserver 25.1.0 and 25.0.x XLibre Drivers in FreeBSD Ports.
https://ports.freebsd.org/cgi/ports.cgi?query=xlibre&sektion=x11

WebZFS is ZFS Web Management Interface for FreeBSD/Linux.
https://github.com/webzfs/webzfs

Hardware

AMD Ryzen 9 9950X3D2 with 192MB L3 Cache Spotted in Multiple Benchmarks.
https://hothardware.com/news/amd-9950x3d2-benchmark-leaks

Raspberry Pi 5 Expansion with 14 USB Ports.
https://cnx-software.com/2025/12/20/do-you-need-14-usb-ports-on-your-raspberry-pi-5/

I am Returning My Framework 16.
https://yorickpeterse.com/articles/im-returning-my-framework-16/

New Radeon RX 9060 XT LP Quietly Launched: 16 GB with Lower TDP.
https://hwcooling.net/en/new-radeon-rx-9060-xt-lp-quietly-launched-16-gb-with-lower-tdp/

Untold Story of Nintendo Entertainment System.
https://youtube.com/watch?v=uJvpRGibFhg

New in Collection – Part 1 – Keyport 717.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-1-keyport-717/

New in Collection – Part 2 – NEC PWP-100.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-2-nec-pwp-100/

New in Collection – Part 3 – Canon PW-10/15/30.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-3-canon-pw-101530/

New in Collection – Part 4 – Non Canon Cats.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-4-non-canon-cats/

Golden Age of Personal Computers – 80 Machines Reviewed.
https://youtube.com/watch?v=wQPmPdL5t3Y

Short History of Personal Computers – 25 Years at Glance.
https://youtube.com/watch?v=JdZ4XFEZFnc

Life

Backing Up Spotify – Distributed in Bulk Torrents (300TB) and Grouped by Popularity.
https://annas-archive.li/blog/backing-up-spotify.html

Why Young Women are Becoming More Liberal Than Young Men – Gender Equality Paradox.
https://robkhenderson.com/p/the-gender-equality-paradox-explains

Cloudflare Says It Does Not Censor Internet – Numbers Tell More Complicated Story.
https://nerds.xyz/2025/12/cloudflare-transparency-report-2025/

Other

Firefox Promises AI Kill Switch.
https://nerds.xyz/2025/12/firefox-ai-kill-switch/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Brave New PKGBASE World

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

Let me put this into first sentence to make things clear – I like PKGBASE and I think it is improvement over freebsd-update(8) and base.txz and kernel.txz – what we currently have in FreeBSD. All the issues will be resolved in time and if You want to play safe you can still use the classic way of FreeBSD life over entire 15.x FreeBSD line.

The Table of Contents for the article.

  • Divided
  • Discussions
  • Problem to Solve
  • Warning
  • Install
  • Documentation
  • New PKGBASE Distribution Sets
  • Minimal RAM Requirements
  • Base Install
  • Repositories
  • Updating
  • Vital
  • PKGBASE Jails
  • Additional Independent Rescue
  • In the Works …

All of the information here is based on the FreeBSD 15.0-BETA2 version but I will update the info as new things are introduced.

Also … as now the base.txz is spread over about 200 or so pkg(8) packages – you will be able to either install everything – like with base.txz or install and maintain only parts that you really need. Do not need compilers? Remove them. Do not want to have documentation/man pages/examples? Uninstall.

Its just important to remember that after you switch any of your FreeBSD systems to PKGBASE – to also switch mentally – to not do several things that you have done in the past in the old ‘classic’ world.

Now …

Divided

I do not remember last time when entire FreeBSD community was more divided in one single concept.

The PKGBASE concept.

FreeBSD concept with which everything is handled by one pkg(8) command – for both Base System and third party packages.

Similar to dnf(8) or apt(8) from the Linux world.

Some time ago I even made a poll on both X/Twitter and Bluesky platforms.

The question was:

– one pkg(8) command for FreeBSD Base System and third party packages.

– separate pkgbase(8) command for FreeBSD Base System and pkg(8) for third party packages.

Results are below.

More or less 50% for each option.

Discussions

I also do not remember when single topic covered most of the Mailing Lists discussions with such coverage.

Below are screenshots or freebsd-pkgbase and freebsd-stable lists.

While I like the PKGBASE concept because now the freebsd-update(8) process is very long over large upgrades and also very interactive – besides other things PKGBASE also solves these two … but it comes at a price.

Problem to Solve

Some may ask what problem PKGBASE tries to resolve … there are few.

First – the delta based design of freebsd-update(8) was PITA for maintenance. The binary diffs where difficult to generate and maintain because FreeBSD Release Engineering team need to store every old version of binaries – then generating binary diffs for each possible upgrade path and later testing that every delta sequence applies correctly. Besides being error prone it was also time consuming. This ‘way’ of things was also sensitive to local modifications and/or corruption. The freebsd-update(8) requires that every local file exactly matches the expected original. If user compiled custom kernel/world or modified/replaced any file manually then the delta patches cannot be applied.

Second – the freebsd-update(8) process was ALWAYS interactive which was quite OK if you have two FreeBSD machines – but it You are responsible for keeping hundreds of FreeBSD machines and need to patch them quickly then it was PITA. Some people coped with that by overwriting the PAGER variable … but that often caused trouble.

FreeBSD # env PAGER=cat freebsd-update fetch install

Third – no file ownership within Base System. If file is not known to pkg(8) then it probably originated in FreeBSD Base System … but you can never be sure. Because freebsd-update(8) only patches files in place it can not track what component owns which file and thus cleanly remove obsolete files.

Forth – with PKGBASE its possible to remove and/or manage optional components cleanly – by removing their pkg(8) packages. Without PKGBASE you can only mange Base System as a whole and patch it with freebsd-update(8) … you can of course build your tailored FreeBSD version with lots of components/subsystems disabled in /etc/src.conf file … but then freebsd-update(8) will not work with them.

Fifth – without PKGBASE – the only way you can update/upgrade a STABLE or CURRENT system is by compiling everything from source. With PKGBASE there will be pkg(8) repos with weekly/monthly updates – so You do not have to waste time and electricity just to update your STABLE/CURRENT system – just update the PKGBASE packages with pkg(8) command.

Warning

I need to warn you about a thing or two in the new PKGBASE world.

[ WARNING: BEGIN ]

Many FreeBSD seasoned sysadmins – including me – knew that from time to time – when its needed – one can just safely wipe all third party packages with pkg delete -fay command. Then one could rm -rf /usr/local /boot/modules to make sure everything was cleaned – and start again with fresh FreeBSD Base System.

With FreeBSD installed in a PKGBASE way the pkg delete -fay command will destroy your FreeBSD system. Literally. So think twice before executing it on a FreeBSD PKGBASE system.

The ZFS Boot Environments feature will NOT protect you from executing pkg delete -fay command on the running system – even if you have created a backup ZFS Boot Environment – you will NOT be able to reach the loader(8) for the boot menu selection.

OK ls boot
boot
 d  zfs
 d  efi
    loader.conf
    entropy
 d  firmware

Because there is not kernel or loader(8) anymore … and its the same in both BIOS and UEFI mode.

Another warning – if you use FreeBSD in PKGBASE mode – do not touch freebsd-update(8) command – it will break the system.

Also – do not update/upgrade PKGBASE system with make installworld or make installkernel commands – it will also do harm.

Alternatively use make buildworld buildkernel update-packages command which will create package repository with packages you can use to update/upgrade the FreeBSD system with pkg(8) command.

This is the default FreeBSD ZFS pool layout.

root@pkgbase:~ # zfs list
NAME                  USED  AVAIL  REFER  MOUNTPOINT
zroot                1.25G  17.7G    96K  /zroot
zroot/ROOT           1.25G  17.7G    96K  none
zroot/ROOT/default   1.24G  17.7G   854M  /
zroot/home            224K  17.7G    96K  /home
zroot/home/vermaden   128K  17.7G   128K  /home/vermaden
zroot/tmp             112K  17.7G   112K  /tmp
zroot/usr             288K  17.7G    96K  /usr
zroot/usr/ports        96K  17.7G    96K  /usr/ports
zroot/usr/src          96K  17.7G    96K  /usr/src
zroot/var             684K  17.7G    96K  /var
zroot/var/audit        96K  17.7G    96K  /var/audit
zroot/var/crash        96K  17.7G    96K  /var/crash
zroot/var/log         196K  17.7G   196K  /var/log
zroot/var/mail        104K  17.7G   104K  /var/mail
zroot/var/tmp          96K  17.7G    96K  /var/tmp

Its a good template to start – I very rarely modify it – but …

If you use multiple independent ZFS Boot Environments keep in mind that /usr/src is not part of ZFS BE and will be overwritten by FreeBSD-src and FreeBSD-src-sys packages from each ZFS BE. To overcome move /usr/src into each ZFS BE so that FreeBSD source tree (and packages) will be consistent and independent from each other for each ZFS Boot Environment you use. Its the same even without PKGBASE – its just a reminder that PKGBASE does not change anything here.

[ WARNING: END ]

Install

Right now bsdinstall from FreeBSD 15.x will ask you important question at the start – in which world you want to live?

I have also read on Mailing Lists that Experimental label will be changed to Tech Preview one.

There is also one important switch on what is available on the installation media. From 15.x series the PKGBASE package sets are on the installation media and older ones like base.txz or kernel.tzx have to be downloaded from the Internet … at least for smallest disc1 media.

Classic Distributions Sets like base.txz and kernel.txz maintained and updated by freebsd-update(8) command or PKGBASE with everything being managed and upgraded by pkg(8) command.

You can even later convert older Distributions Sets install with pkgbasify(8) tool. From what I have seen on the FreeBSD Mailing Lists there is even depkgbasify(8) planned – to convert PKGBASE system back to classic Distributions Sets setup.

Documentation

After VNET Jails were introduced to FreeBSD the official FreeBSD Handbook did not had any documentation about them for more then a decade.

Fortunately the PKGBASE concept is already covered FreeBSD Handbook PKGBASE section.

… and its not some short memo/mention like with fwget(8) tool that was just mentioned once in the FreeBSD 14.0-RELEASE Release Notes and forgotten.

There is also new freebsd-base(7) man page covering PKGBASE concept.

Which is also nice.

There is also some relevant info in the FreeBSD Wiki https://wiki.freebsd.org/pkgbase page.

New PKGBASE Distribution Sets

Say goodbye to traditional base.txz and kernel.tzx sets and welcome newly crafted sets for PKGBASE:

kernel-dbg – Debug symbols for the kernel.

devel – C/C++ compilers and related utilities.

optional – Optional software (excluding compilers).

optional-jail – Optional software (excluding compilers)

base – The complete base system (includes devel and optional)

base-jail – The complete base system (includes devel and optional)

src – System source tree

tests – Test suite.

lib32 – 32-bit compatibility libraries.

debug – Debug symbols for the selected components.

From the good news – you can now select to install a lot smaller Base System that is a lot better suited for FreeBSD Jails – and there are no bad news here – just more options and flexibility 🙂

Minimal RAM Requirements

I am not sure that there are any specific requirements set right now – but the minimum amount of RAM that I was able to install FreeBSD with PKGBASE is 300 MB RAM – and that is with Auto (ZFS) option. I already described it in the Mailing Lists but I will happily repeat that here.

First Select [Live System] option – then execute these:

root@:~ # kldload zfs
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)

root@:~ # sysctl vfs.zfs.arc.min=33554432
vfs.zfs.arc.min: 0 -> 33554432

root@:~ # sysctl vfs.zfs.arc.max=67108864
vfs.zfs.arc.max: 0 -> 67108864

root@:~ # sysctl vm.pageout_oom_seq=1200
vm.pageout_oom_seq: 12 -> 1200

root@:~ # env TERM=xterm bsdinstall

Now you can proceed to the install process. I selected PKGBASE type and offline installation – and then Auto (ZFS). I made the test with only base set chosen – but it probably can also survive more sets selected. I also switched from (BIOS) to (BIOS+UEFI) but that should not make any difference for you.

Some stats after installation (before reboot):

root@:/ # dmesg | grep -i memory
real memory  = 314572800 (300 MB)
avail memory = 263303168 (251 MB)

root@:/ # top -b -o res
last pid:  2544;  load averages:    1.31,    0.80,    0.34  up 0+00:02:20    18:02:03
10 processes:  1 running, 9 sleeping
CPU:  7.8% user,  0.0% nice, 16.7% system, 12.1% interrupt, 63.4% idle
Mem: 3588K Active, 11M Inact, 6376K Laundry, 148M Wired, 5800K Buf, 89M Free
ARC: 47M Total, 2667K MFU, 38M MRU, 128K Anon, 294K Header, 6515K Other
     29M Compressed, 65M Uncompressed, 2.24:1 Ratio

I tried the same with 256 MB RAM but it fails.

Base Install

After selecting base only I got 208 packages installed – including pkg(8) one.

root@pkgbase:~ # pkg info | wc -l
     208

root@pkgbase:~ # pkg info | while read PKG DESC; do printf "%52s  %s\n" ${PKG} ${DESC}; done
                 FreeBSD-acct-15.0.b1.20251011075131  System resource accounting
                 FreeBSD-acpi-15.0.b1.20251011075131  Advanced Configuration and Power Interface (ACPI) utilities
                  FreeBSD-apm-15.0.b1.20251011075131  Intel / Microsoft APM BIOS utility
                   FreeBSD-at-15.0.b1.20251011075131  Scheduled and batch command utilities
                  FreeBSD-atf-15.0.b1.20251011075131  Automated Testing Framework
              FreeBSD-atf-dev-15.0.b1.20251011075131  Automated Testing Framework (development files)
              FreeBSD-atf-lib-15.0.b1.20251011075131  Automated Testing Framework (libraries)
                FreeBSD-audit-15.0.b1.20251011075131  OpenBSM auditing utilities
            FreeBSD-audit-dev-15.0.b1.20251011075131  OpenBSM auditing utilities (development files)
            FreeBSD-audit-lib-15.0.b1.20251011075131  OpenBSM auditing utilities (libraries)
               FreeBSD-autofs-15.0.b1.20251011075131  File system automounter
                FreeBSD-bhyve-15.0.b1.20251011075131  bhyve virtual machine hypervisor
            FreeBSD-blocklist-15.0.b1.20251011075131  Network blocklist daemon
        FreeBSD-blocklist-dev-15.0.b1.20251011075131  Network blocklist daemon (development files)
            FreeBSD-bluetooth-15.0.b1.20251011075131  Bluetooth support
        FreeBSD-bluetooth-dev-15.0.b1.20251011075131  Bluetooth support (development files)
        FreeBSD-bluetooth-lib-15.0.b1.20251011075131  Bluetooth support (libraries)
                FreeBSD-bmake-15.0.b1.20251011075131  Program maintenance utility
           FreeBSD-bootloader-15.0.b1.20251011075131  System boot loader
       FreeBSD-bootloader-dev-15.0.b1.20251011075131  System boot loader (development files)
            FreeBSD-bsdconfig-15.0.b1.20251011075131  System configuration utility
           FreeBSD-bsdinstall-15.0.b1.20251011075131  System installer
                FreeBSD-bsnmp-15.0.b1.20251011075131  Simple and extensible SNMP daemon
            FreeBSD-bsnmp-dev-15.0.b1.20251011075131  Simple and extensible SNMP daemon (development files)
                FreeBSD-bzip2-15.0.b1.20251011075131  A block-sorting data compressor
            FreeBSD-bzip2-dev-15.0.b1.20251011075131  A block-sorting data compressor (development files)
            FreeBSD-bzip2-lib-15.0.b1.20251011075131  A block-sorting data compressor (libraries)
               FreeBSD-caroot-15.0.b1.20251011075131  Mozilla Root Store trusted TLS certificates
            FreeBSD-ccdconfig-15.0.b1.20251011075131  Concatenated disk driver (ccd) configuration utility
              FreeBSD-certctl-15.0.b1.20251011075131  Tool for managing trusted and untrusted TLS certificates
                FreeBSD-clang-15.0.b1.20251011075131  The Clang C, C++, and Objective-C compiler
            FreeBSD-clang-dev-15.0.b1.20251011075131  The Clang C, C++, and Objective-C compiler (development files)
                FreeBSD-clibs-15.0.b1.20251011075131  Core runtime libraries
            FreeBSD-clibs-dev-15.0.b1.20251011075131  Core runtime libraries (development files)
        FreeBSD-console-tools-15.0.b1.20251011075131  Video console utilities
                 FreeBSD-cron-15.0.b1.20251011075131  Command scheduling facility
                  FreeBSD-csh-15.0.b1.20251011075131  C shell with file name completion and command line editing
                  FreeBSD-ctf-15.0.b1.20251011075131  Compact C Type Format (CTF)
              FreeBSD-ctf-dev-15.0.b1.20251011075131  Compact C Type Format (CTF) (development files)
              FreeBSD-ctf-lib-15.0.b1.20251011075131  Compact C Type Format (CTF) (libraries)
                  FreeBSD-ctl-15.0.b1.20251011075131  CAM Target Layer / iSCSI target daemon
          FreeBSD-cxgbe-tools-15.0.b1.20251011075131  Configuration utility for Chelsio cxbge(4) network interfaces
                 FreeBSD-devd-15.0.b1.20251011075131  Kernel event handling daemon
             FreeBSD-devmatch-15.0.b1.20251011075131  Automatically load kernel drivers for attached hardware
         FreeBSD-devmatch-dev-15.0.b1.20251011075131  Automatically load kernel drivers for attached hardware (development files)
             FreeBSD-dhclient-15.0.b1.20251011075131  Dynamic Host Configuration Protocol (DHCP) client
                FreeBSD-diff3-15.0.b1.20251011075131  GNU 3-way file comparison and merge utility
                  FreeBSD-dma-15.0.b1.20251011075131  DragonFly Mail Agent
               FreeBSD-dtrace-15.0.b1.20251011075131  DTrace dynamic tracing framework
           FreeBSD-dtrace-dev-15.0.b1.20251011075131  DTrace dynamic tracing framework (development files)
               FreeBSD-dwatch-15.0.b1.20251011075131  Watch processes as they trigger a particular DTrace probe
                   FreeBSD-ee-15.0.b1.20251011075131  Easy Editor
            FreeBSD-efi-tools-15.0.b1.20251011075131  Utilities for the Unified Extensible Firmware Interface (UEFI)
        FreeBSD-efi-tools-dev-15.0.b1.20251011075131  Utilities for the Unified Extensible Firmware Interface (UEFI) (development files)
             FreeBSD-examples-15.0.b1.20251011075131  System interface examples
                   FreeBSD-fd-15.0.b1.20251011075131  Floppy disk utilities
                FreeBSD-fetch-15.0.b1.20251011075131  Retrieve a file from a remote URL
            FreeBSD-fetch-dev-15.0.b1.20251011075131  Retrieve a file from a remote URL (development files)
         FreeBSD-firmware-iwm-15.0.b1.20251011075131  Firmware for iwm(4) Intel 802.11ac network interfaces
                  FreeBSD-ftp-15.0.b1.20251011075131  File Transfer Protocol (FTP) client
                FreeBSD-fwget-15.0.b1.20251011075131  Install firmware packages for the running system
                FreeBSD-games-15.0.b1.20251011075131  Games
                 FreeBSD-geom-15.0.b1.20251011075131  GEOM configuration utilities
                FreeBSD-ggate-15.0.b1.20251011075131  GEOM Gate client and server
                 FreeBSD-gssd-15.0.b1.20251011075131  gssd(8) daemon for kernel GSS-API
                 FreeBSD-hast-15.0.b1.20251011075131  Highly Available Storage daemon
              FreeBSD-hostapd-15.0.b1.20251011075131  IEEE Std 802.11 Access Point authentication daemon
         FreeBSD-hyperv-tools-15.0.b1.20251011075131  Microsoft Hyper-V utilities
                FreeBSD-inetd-15.0.b1.20251011075131  Internet super-server
                  FreeBSD-ipf-15.0.b1.20251011075131  IP Filter (ipf) packet filter management tools
                 FreeBSD-ipfw-15.0.b1.20251011075131  ipfw (IP firewall) management utilities
                FreeBSD-iscsi-15.0.b1.20251011075131  iSCSI initiator
                 FreeBSD-jail-15.0.b1.20251011075131  Jail management tools
             FreeBSD-kerberos-15.0.b1.20251011075131  Kerberos client
         FreeBSD-kerberos-dev-15.0.b1.20251011075131  Kerberos client (development files)
         FreeBSD-kerberos-kdc-15.0.b1.20251011075131  Kerberos key distribution center
         FreeBSD-kerberos-lib-15.0.b1.20251011075131  Kerberos client (libraries)
       FreeBSD-kernel-generic-15.0.b1.20251011075131  FreeBSD GENERIC Kernel
           FreeBSD-kernel-man-15.0.b1.20251011075131  Kernel manual pages
                 FreeBSD-kyua-15.0.b1.20251011075131  Testing framework for infrastructure software
                FreeBSD-lib9p-15.0.b1.20251011075131  9P network protocol library
            FreeBSD-lib9p-dev-15.0.b1.20251011075131  9P network protocol library (development files)
           FreeBSD-libarchive-15.0.b1.20251011075131  Library for reading and writing streaming archives
       FreeBSD-libarchive-dev-15.0.b1.20251011075131  Library for reading and writing streaming archives (development files)
           FreeBSD-libbegemot-15.0.b1.20251011075131  rpoll(3) interface for event-driven I/O
       FreeBSD-libbegemot-dev-15.0.b1.20251011075131  rpoll(3) interface for event-driven I/O (development files)
     FreeBSD-libblocksruntime-15.0.b1.20251011075131  LLVM BlocksRuntime library
 FreeBSD-libblocksruntime-dev-15.0.b1.20251011075131  LLVM BlocksRuntime library (development files)
           FreeBSD-libbsdstat-15.0.b1.20251011075131  Periodic statistics library
       FreeBSD-libbsdstat-dev-15.0.b1.20251011075131  Periodic statistics library (development files)
            FreeBSD-libcasper-15.0.b1.20251011075131  Casper library
        FreeBSD-libcasper-dev-15.0.b1.20251011075131  Casper library (development files)
            FreeBSD-libcompat-15.0.b1.20251011075131  Compatibility library
        FreeBSD-libcompat-dev-15.0.b1.20251011075131  Compatibility library (development files)
   FreeBSD-libcompiler_rt-dev-15.0.b1.20251011075131  LLVM compiler_rt library (development files)
              FreeBSD-libcuse-15.0.b1.20251011075131  Userland character device library
          FreeBSD-libcuse-dev-15.0.b1.20251011075131  Userland character device library (development files)
             FreeBSD-libdwarf-15.0.b1.20251011075131  DWARF access library
         FreeBSD-libdwarf-dev-15.0.b1.20251011075131  DWARF access library (development files)
            FreeBSD-libevent1-15.0.b1.20251011075131  Private libevent1 library
        FreeBSD-libevent1-dev-15.0.b1.20251011075131  Private libevent1 library (development files)
          FreeBSD-libexecinfo-15.0.b1.20251011075131  NetBSD stack backtrace library
      FreeBSD-libexecinfo-dev-15.0.b1.20251011075131  NetBSD stack backtrace library (development files)
               FreeBSD-libipt-15.0.b1.20251011075131  Intel(R) Processor Trace decoder library
           FreeBSD-libipt-dev-15.0.b1.20251011075131  Intel(R) Processor Trace decoder library (development files)
              FreeBSD-libldns-15.0.b1.20251011075131  Private LDNS library
          FreeBSD-libldns-dev-15.0.b1.20251011075131  Private LDNS library (development files)
             FreeBSD-libmagic-15.0.b1.20251011075131  Magic number recognition library
         FreeBSD-libmagic-dev-15.0.b1.20251011075131  Magic number recognition library (development files)
            FreeBSD-libmilter-15.0.b1.20251011075131  sendmail Mail Filter API library
        FreeBSD-libmilter-dev-15.0.b1.20251011075131  sendmail Mail Filter API library (development files)
          FreeBSD-libpathconv-15.0.b1.20251011075131  Library for handling relative and absolute pathnames
      FreeBSD-libpathconv-dev-15.0.b1.20251011075131  Library for handling relative and absolute pathnames (development files)
        FreeBSD-librpcsec_gss-15.0.b1.20251011075131  RPCSEC_GSS library
    FreeBSD-librpcsec_gss-dev-15.0.b1.20251011075131  RPCSEC_GSS library (development files)
               FreeBSD-librss-15.0.b1.20251011075131  Receive-side scaling library
           FreeBSD-librss-dev-15.0.b1.20251011075131  Receive-side scaling library (development files)
           FreeBSD-libsqlite3-15.0.b1.20251011075131  Private SQLite library
       FreeBSD-libsqlite3-dev-15.0.b1.20251011075131  Private SQLite library (development files)
         FreeBSD-libthread_db-15.0.b1.20251011075131  Library for interacting with threaded processes
     FreeBSD-libthread_db-dev-15.0.b1.20251011075131  Library for interacting with threaded processes (development files)
               FreeBSD-libucl-15.0.b1.20251011075131  Private Universal Configuration Library (UCL) library
           FreeBSD-libucl-dev-15.0.b1.20251011075131  Private Universal Configuration Library (UCL) library (development files)
               FreeBSD-libvgl-15.0.b1.20251011075131  Video Graphics Library
           FreeBSD-libvgl-dev-15.0.b1.20251011075131  Video Graphics Library (development files)
            FreeBSD-libvmmapi-15.0.b1.20251011075131  Front-end library for the vmm(4) virtualization driver
        FreeBSD-libvmmapi-dev-15.0.b1.20251011075131  Front-end library for the vmm(4) virtualization driver (development files)
              FreeBSD-libyaml-15.0.b1.20251011075131  Private YAML library
          FreeBSD-libyaml-dev-15.0.b1.20251011075131  Private YAML library (development files)
                  FreeBSD-lld-15.0.b1.20251011075131  LLVM ELF link loader
                 FreeBSD-lldb-15.0.b1.20251011075131  LLVM debugger
             FreeBSD-lldb-dev-15.0.b1.20251011075131  LLVM debugger (development files)
              FreeBSD-locales-15.0.b1.20251011075131  Locale definitions
                   FreeBSD-lp-15.0.b1.20251011075131  Print spooler
               FreeBSD-mandoc-15.0.b1.20251011075131  Online manual page reader
            FreeBSD-mlx-tools-15.0.b1.20251011075131  Utility for managing Connect-X 4/5/6 Mellanox network adapters
                FreeBSD-mtree-15.0.b1.20251011075131  Filesystem hierarchy management tool
                 FreeBSD-natd-15.0.b1.20251011075131  Userland Network Address Translation (NAT) for ipfw
             FreeBSD-natd-dev-15.0.b1.20251011075131  Userland Network Address Translation (NAT) for ipfw (development files)
              FreeBSD-ncurses-15.0.b1.20251011075131  ncurses terminal control library
          FreeBSD-ncurses-dev-15.0.b1.20251011075131  ncurses terminal control library (development files)
               FreeBSD-netmap-15.0.b1.20251011075131  Userland netmap support
           FreeBSD-netmap-dev-15.0.b1.20251011075131  Userland netmap support (development files)
            FreeBSD-newsyslog-15.0.b1.20251011075131  Create and rotate system log files
                  FreeBSD-nfs-15.0.b1.20251011075131  NFS client and server utilities
                  FreeBSD-ntp-15.0.b1.20251011075131  Network Time Protocol (NTP) daemon
            FreeBSD-nuageinit-15.0.b1.20251011075131  cloud-init configuration support
           FreeBSD-nvme-tools-15.0.b1.20251011075131  NVMe utilities
              FreeBSD-openssl-15.0.b1.20251011075131  OpenSSL Transport Layer Security (TLS) library
          FreeBSD-openssl-dev-15.0.b1.20251011075131  OpenSSL Transport Layer Security (TLS) library (development files)
          FreeBSD-openssl-lib-15.0.b1.20251011075131  OpenSSL Transport Layer Security (TLS) library (libraries)
             FreeBSD-periodic-15.0.b1.20251011075131  Run periodic system functions
                   FreeBSD-pf-15.0.b1.20251011075131  OpenBSD packet filter
               FreeBSD-pf-dev-15.0.b1.20251011075131  OpenBSD packet filter (development files)
        FreeBSD-pkg-bootstrap-15.0.b1.20251011075131  pkg(7) bootstrap utility
                  FreeBSD-pmc-15.0.b1.20251011075131  Support for hardware performance counters
              FreeBSD-pmc-dev-15.0.b1.20251011075131  Support for hardware performance counters (development files)
               FreeBSD-powerd-15.0.b1.20251011075131  System power control utility
                  FreeBSD-ppp-15.0.b1.20251011075131  Point-to-Point Protocol (PPP) utilities
           FreeBSD-quotacheck-15.0.b1.20251011075131  Filesystem quota consistency checker
                   FreeBSD-rc-15.0.b1.20251011075131  rc(8) subsystem
                FreeBSD-rcmds-15.0.b1.20251011075131  BSD/SunOS remote status commands
                 FreeBSD-rdma-15.0.b1.20251011075131  Remote Direct Memory Access (RDMA) and InfiniBand utilities
               FreeBSD-rescue-15.0.b1.20251011075131  Rescue system
           FreeBSD-resolvconf-15.0.b1.20251011075131  A framework for managing multiple DNS configurations
                  FreeBSD-rip-15.0.b1.20251011075131  RIP routing protocol
              FreeBSD-runtime-15.0.b1.20251011075131  Core system
          FreeBSD-runtime-dev-15.0.b1.20251011075131  Core system (development files)
             FreeBSD-sendmail-15.0.b1.20251011075131  sendmail mail transport agent
             FreeBSD-set-base-15.0.b1.20251011075131  Base system (metapackage)
            FreeBSD-set-devel-15.0.b1.20251011075131  Development tools (metapackage)
          FreeBSD-set-minimal-15.0.b1.20251011075131  Basic multi-user system (metapackage)
         FreeBSD-set-optional-15.0.b1.20251011075131  Optional base system software (metapackage)
              FreeBSD-set-src-15.0.b1.20251011075131  System source code
             FreeBSD-smbutils-15.0.b1.20251011075131  SMB network filesystem utilities
         FreeBSD-smbutils-dev-15.0.b1.20251011075131  SMB network filesystem utilities (development files)
                FreeBSD-sound-15.0.b1.20251011075131  Audio utilities
            FreeBSD-sound-dev-15.0.b1.20251011075131  Audio utilities (development files)
                  FreeBSD-src-15.0.b1.20251011075131  System userland source code
              FreeBSD-src-sys-15.0.b1.20251011075131  System kernel source code
                  FreeBSD-ssh-15.0.b1.20251011075131  OpenSSH Secure Shell client and server
              FreeBSD-ssh-dev-15.0.b1.20251011075131  OpenSSH Secure Shell client and server (development files)
         FreeBSD-syscons-data-15.0.b1.20251011075131  syscons(4) fonts and keymaps
              FreeBSD-syslogd-15.0.b1.20251011075131  System logging daemon
                 FreeBSD-tcpd-15.0.b1.20251011075131  TCP Wrappers access control facility
             FreeBSD-tcpd-dev-15.0.b1.20251011075131  TCP Wrappers access control facility (development files)
               FreeBSD-telnet-15.0.b1.20251011075131  TELNET client
            FreeBSD-toolchain-15.0.b1.20251011075131  Utilities for program development
        FreeBSD-toolchain-dev-15.0.b1.20251011075131  Utilities for program development (development files)
                  FreeBSD-ufs-15.0.b1.20251011075131  UFS filesystem support
              FreeBSD-ufs-dev-15.0.b1.20251011075131  UFS filesystem support (development files)
              FreeBSD-ufs-lib-15.0.b1.20251011075131  UFS filesystem support (libraries)
              FreeBSD-unbound-15.0.b1.20251011075131  NLnet Labs Unbound DNS resolver
          FreeBSD-unbound-dev-15.0.b1.20251011075131  NLnet Labs Unbound DNS resolver (development files)
            FreeBSD-utilities-15.0.b1.20251011075131  Non-vital programs and libraries
        FreeBSD-utilities-dev-15.0.b1.20251011075131  Non-vital programs and libraries (development files)
                   FreeBSD-vi-15.0.b1.20251011075131  Text editor
              FreeBSD-vt-data-15.0.b1.20251011075131  vt(4) fonts and keymaps
                  FreeBSD-wpa-15.0.b1.20251011075131  IEEE Std 802.11 WPA Supplicant
                   FreeBSD-xz-15.0.b1.20251011075131  LZMA2 data compression
               FreeBSD-xz-dev-15.0.b1.20251011075131  LZMA2 data compression (development files)
               FreeBSD-xz-lib-15.0.b1.20251011075131  LZMA2 data compression (libraries)
                   FreeBSD-yp-15.0.b1.20251011075131  Yellow Pages (YP) / Network Information Service (NIS)
                  FreeBSD-zfs-15.0.b1.20251011075131  ZFS filesystem support
              FreeBSD-zfs-dev-15.0.b1.20251011075131  ZFS filesystem support (development files)
              FreeBSD-zfs-lib-15.0.b1.20251011075131  ZFS filesystem support (libraries)
             FreeBSD-zoneinfo-15.0.b1.20251011075131  Timezone database
                                           pkg-2.3.1  Package manager

root@pkgbase:~ # pkg info -qoa
base/FreeBSD-acct
base/FreeBSD-acpi
base/FreeBSD-apm
base/FreeBSD-at
base/FreeBSD-atf
base/FreeBSD-atf-dev
base/FreeBSD-atf-lib
base/FreeBSD-audit
base/FreeBSD-audit-dev
base/FreeBSD-audit-lib
base/FreeBSD-autofs
base/FreeBSD-bhyve
base/FreeBSD-blocklist
base/FreeBSD-blocklist-dev
base/FreeBSD-bluetooth
base/FreeBSD-bluetooth-dev
base/FreeBSD-bluetooth-lib
base/FreeBSD-bmake
base/FreeBSD-bootloader
base/FreeBSD-bootloader-dev
base/FreeBSD-bsdconfig
base/FreeBSD-bsdinstall
base/FreeBSD-bsnmp
base/FreeBSD-bsnmp-dev
base/FreeBSD-bzip2
base/FreeBSD-bzip2-dev
base/FreeBSD-bzip2-lib
base/FreeBSD-caroot
base/FreeBSD-ccdconfig
base/FreeBSD-certctl
base/FreeBSD-clang
base/FreeBSD-clang-dev
base/FreeBSD-clibs
base/FreeBSD-clibs-dev
base/FreeBSD-console-tools
base/FreeBSD-cron
base/FreeBSD-csh
base/FreeBSD-ctf
base/FreeBSD-ctf-dev
base/FreeBSD-ctf-lib
base/FreeBSD-ctl
base/FreeBSD-cxgbe-tools
base/FreeBSD-devd
base/FreeBSD-devmatch
base/FreeBSD-devmatch-dev
base/FreeBSD-dhclient
base/FreeBSD-diff3
base/FreeBSD-dma
base/FreeBSD-dtrace
base/FreeBSD-dtrace-dev
base/FreeBSD-dwatch
base/FreeBSD-ee
base/FreeBSD-efi-tools
base/FreeBSD-efi-tools-dev
base/FreeBSD-examples
base/FreeBSD-fd
base/FreeBSD-fetch
base/FreeBSD-fetch-dev
base/FreeBSD-firmware-iwm
base/FreeBSD-ftp
base/FreeBSD-fwget
base/FreeBSD-games
base/FreeBSD-geom
base/FreeBSD-ggate
base/FreeBSD-gssd
base/FreeBSD-hast
base/FreeBSD-hostapd
base/FreeBSD-hyperv-tools
base/FreeBSD-inetd
base/FreeBSD-ipf
base/FreeBSD-ipfw
base/FreeBSD-iscsi
base/FreeBSD-jail
base/FreeBSD-kerberos
base/FreeBSD-kerberos-dev
base/FreeBSD-kerberos-kdc
base/FreeBSD-kerberos-lib
base/FreeBSD-kernel-generic
base/FreeBSD-kernel-man
base/FreeBSD-kyua
base/FreeBSD-lib9p
base/FreeBSD-lib9p-dev
base/FreeBSD-libarchive
base/FreeBSD-libarchive-dev
base/FreeBSD-libbegemot
base/FreeBSD-libbegemot-dev
base/FreeBSD-libblocksruntime
base/FreeBSD-libblocksruntime-dev
base/FreeBSD-libbsdstat
base/FreeBSD-libbsdstat-dev
base/FreeBSD-libcasper
base/FreeBSD-libcasper-dev
base/FreeBSD-libcompat
base/FreeBSD-libcompat-dev
base/FreeBSD-libcompiler_rt-dev
base/FreeBSD-libcuse
base/FreeBSD-libcuse-dev
base/FreeBSD-libdwarf
base/FreeBSD-libdwarf-dev
base/FreeBSD-libevent1
base/FreeBSD-libevent1-dev
base/FreeBSD-libexecinfo
base/FreeBSD-libexecinfo-dev
base/FreeBSD-libipt
base/FreeBSD-libipt-dev
base/FreeBSD-libldns
base/FreeBSD-libldns-dev
base/FreeBSD-libmagic
base/FreeBSD-libmagic-dev
base/FreeBSD-libmilter
base/FreeBSD-libmilter-dev
base/FreeBSD-libpathconv
base/FreeBSD-libpathconv-dev
base/FreeBSD-librpcsec_gss
base/FreeBSD-librpcsec_gss-dev
base/FreeBSD-librss
base/FreeBSD-librss-dev
base/FreeBSD-libsqlite3
base/FreeBSD-libsqlite3-dev
base/FreeBSD-libthread_db
base/FreeBSD-libthread_db-dev
base/FreeBSD-libucl
base/FreeBSD-libucl-dev
base/FreeBSD-libvgl
base/FreeBSD-libvgl-dev
base/FreeBSD-libvmmapi
base/FreeBSD-libvmmapi-dev
base/FreeBSD-libyaml
base/FreeBSD-libyaml-dev
base/FreeBSD-lld
base/FreeBSD-lldb
base/FreeBSD-lldb-dev
base/FreeBSD-locales
base/FreeBSD-lp
base/FreeBSD-mandoc
base/FreeBSD-mlx-tools
base/FreeBSD-mtree
base/FreeBSD-natd
base/FreeBSD-natd-dev
base/FreeBSD-ncurses
base/FreeBSD-ncurses-dev
base/FreeBSD-netmap
base/FreeBSD-netmap-dev
base/FreeBSD-newsyslog
base/FreeBSD-nfs
base/FreeBSD-ntp
base/FreeBSD-nuageinit
base/FreeBSD-nvme-tools
base/FreeBSD-openssl
base/FreeBSD-openssl-dev
base/FreeBSD-openssl-lib
base/FreeBSD-periodic
base/FreeBSD-pf
base/FreeBSD-pf-dev
base/FreeBSD-pkg-bootstrap
base/FreeBSD-pmc
base/FreeBSD-pmc-dev
base/FreeBSD-powerd
base/FreeBSD-ppp
base/FreeBSD-quotacheck
base/FreeBSD-rc
base/FreeBSD-rcmds
base/FreeBSD-rdma
base/FreeBSD-rescue
base/FreeBSD-resolvconf
base/FreeBSD-rip
base/FreeBSD-runtime
base/FreeBSD-runtime-dev
base/FreeBSD-sendmail
base/FreeBSD-set-base
base/FreeBSD-set-devel
base/FreeBSD-set-minimal
base/FreeBSD-set-optional
base/FreeBSD-set-src
base/FreeBSD-smbutils
base/FreeBSD-smbutils-dev
base/FreeBSD-sound
base/FreeBSD-sound-dev
base/FreeBSD-src
base/FreeBSD-src-sys
base/FreeBSD-ssh
base/FreeBSD-ssh-dev
base/FreeBSD-syscons-data
base/FreeBSD-syslogd
base/FreeBSD-tcpd
base/FreeBSD-tcpd-dev
base/FreeBSD-telnet
base/FreeBSD-toolchain
base/FreeBSD-toolchain-dev
base/FreeBSD-ufs
base/FreeBSD-ufs-dev
base/FreeBSD-ufs-lib
base/FreeBSD-unbound
base/FreeBSD-unbound-dev
base/FreeBSD-utilities
base/FreeBSD-utilities-dev
base/FreeBSD-vi
base/FreeBSD-vt-data
base/FreeBSD-wpa
base/FreeBSD-xz
base/FreeBSD-xz-dev
base/FreeBSD-xz-lib
base/FreeBSD-yp
base/FreeBSD-zfs
base/FreeBSD-zfs-dev
base/FreeBSD-zfs-lib
base/FreeBSD-zoneinfo
ports-mgmt/pkg

You can even display pkg(8) distribution sets.

root@pkgbase:~ # pkg info -d -C -x '^FreeBSD-set-' | head
FreeBSD-set-base-15.0.b1.20251011075131:
        FreeBSD-set-optional-15.0.b1.20251011075131
        FreeBSD-set-minimal-15.0.b1.20251011075131
        FreeBSD-set-devel-15.0.b1.20251011075131
FreeBSD-set-devel-15.0.b1.20251011075131:
        FreeBSD-zfs-dev-15.0.b1.20251011075131
        FreeBSD-xz-dev-15.0.b1.20251011075131
        FreeBSD-utilities-dev-15.0.b1.20251011075131
        FreeBSD-unbound-dev-15.0.b1.20251011075131
        FreeBSD-ufs-dev-15.0.b1.20251011075131

root@pkgbase:~ # pkg sets | grep '^FreeBSD'
FreeBSD-set-base-15.0.b1.20251011075131:
FreeBSD-set-devel-15.0.b1.20251011075131:
FreeBSD-set-minimal-15.0.b1.20251011075131:
FreeBSD-set-optional-15.0.b1.20251011075131:
FreeBSD-set-src-15.0.b1.20251011075131:

You can even put it as an ALIAS into the /usr/local/etc/pkg.conf file.

root@pkgbase:~ # grep -e '^ALIAS' -e sets -e '^}' /usr/local/etc/pkg.conf
ALIAS {
    sets = "info -d -C -x '^FreeBSD-set-'";
}

root@pkgbase:~ # pkg sets | head 
FreeBSD-set-base-15.0.b1.20251011075131:
        FreeBSD-set-optional-15.0.b1.20251011075131
        FreeBSD-set-minimal-15.0.b1.20251011075131
        FreeBSD-set-devel-15.0.b1.20251011075131
FreeBSD-set-devel-15.0.b1.20251011075131:
        FreeBSD-zfs-dev-15.0.b1.20251011075131
        FreeBSD-xz-dev-15.0.b1.20251011075131
        FreeBSD-utilities-dev-15.0.b1.20251011075131
        FreeBSD-unbound-dev-15.0.b1.20251011075131
        FreeBSD-ufs-dev-15.0.b1.20251011075131

Repositories

For naming clarity the FreeBSD and FreeBSD-kmods repos that you knew from 14.x line were renamed into FreeBSD-ports and FreeBSD-ports-kmods respectively. The new PKGBASE repo is called FreeBSD-base.

root@pkgbase:~ # pkg repos -l
FreeBSD-ports
FreeBSD-ports-kmods
FreeBSD-base

root@pkgbase:~ # pkg repos
FreeBSD-ports: { 
    url             : "pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/quarterly",
    enabled         : yes,
    priority        : 0,
    mirror_type     : "SRV",
    signature_type  : "FINGERPRINTS",
    fingerprints    : "/usr/share/keys/pkg"
  }
FreeBSD-ports-kmods: { 
    url             : "pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/kmods_quarterly_0",
    enabled         : yes,
    priority        : 0,
    mirror_type     : "SRV",
    signature_type  : "FINGERPRINTS",
    fingerprints    : "/usr/share/keys/pkg"
  }
FreeBSD-base: { 
    url             : "pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/base_latest",
    enabled         : yes,
    priority        : 0,
    mirror_type     : "SRV",
    signature_type  : "FINGERPRINTS",
    fingerprints    : "/usr/share/keys/pkg"
  }

For now the ‘third party’ packages pkg(8) repos config is in the Base System /etc/pkg/FreeBSD.conf file.

root@pkgbase:~ # grep '^[^#]' /etc/pkg/FreeBSD.conf
FreeBSD-ports: {
  url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
FreeBSD-ports-kmods: {
  url: "pkg+https://pkg.FreeBSD.org/${ABI}/kmods_quarterly_${VERSION_MINOR}",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

While the Base System packages pkg(8) repos config is in the ‘third party’ /usr/local/etc/pkg/FreeBSD.conf file.

root@pkgbase:~ # grep '^[^#]' /usr/local/etc/pkg/repos/FreeBSD-base.conf
FreeBSD-base: {
  url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

Which is not logical … but Colin Percival already wrote that this is just temporary and that all of these FreeBSD repos will be in the /etc/pkg/FreeBSD.conf file.

You can update all of them at once or by repo if needed.

root@pkgbase:~ # pkg update
Updating FreeBSD-ports repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   10 MiB 552.2kB/s    00:19    
Processing entries: 100%
FreeBSD-ports repository update completed. 36441 packages processed.
Updating FreeBSD-ports-kmods repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   28 KiB  29.1kB/s    00:01    
Processing entries: 100%
FreeBSD-ports-kmods repository update completed. 199 packages processed.
Updating FreeBSD-base repository catalogue...
pkg: Repository FreeBSD-base has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   76 KiB  77.7kB/s    00:01    
Processing entries:   0%
Newer FreeBSD version for package FreeBSD-zoneinfo:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1500500
- running userland: 1500067
Ignore the mismatch and continue? [y/N]: y
Processing entries: 100%
FreeBSD-base repository update completed. 490 packages processed.
All repositories are up to date.

Updating

To update only the FreeBSD PKGBASE Base System specify the repo in the pkg(8) command.

root@pkgbase:~ # pkg upgrade -r FreeBSD-base
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
FreeBSD-base is up to date.
Checking for upgrades (205 candidates): 100%
Processing candidates (205 candidates): 100%
Checking integrity... done (0 conflicting)
Your packages are up to date.

No updates … strange considering BETA2 is now already available.

After I searched for an answer I got info that You need to modify the URL – to look like that one below.

root@pkgbase:~ # cat /usr/local/etc/pkg/repos/FreeBSD-base.conf
FreeBSD-base: {
# url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_latest",
  url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_release_${VERSION_MINOR}",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

Now lets try to upgrade …

root@pkgbase:~ # pkg update -r FreeBSD-base
Updating FreeBSD-base repository catalogue...
pkg: Repository FreeBSD-base has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   76 KiB  77.7kB/s    00:01    
Processing entries: 100%
FreeBSD-base repository update completed. 490 packages processed.
FreeBSD-base is up to date.

Updating a single Base System package.

root@pkgbase:~ # pkg install FreeBSD-vi
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-ports-kmods repository catalogue...
FreeBSD-ports-kmods repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        FreeBSD-clibs-lib32: 15.0.b1.20251012072228 [FreeBSD-base]

Installed packages to be UPGRADED:
        FreeBSD-vi: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]

Number of packages to be installed: 1
Number of packages to be upgraded: 1

The process will require 4 MiB more space.
2 MiB to be downloaded.

Proceed with this action? [y/N]: y

Updating/upgrading whole FreeBSD system with PKGBASE packages.

root@pkgbase:~ # pkg upgrade -r FreeBSD-base
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
FreeBSD-base is up to date.
Checking for upgrades (202 candidates): 100%
Processing candidates (202 candidates): 100%
Checking integrity... done (5 conflicting)
  - FreeBSD-sound-15.0.b1.20251013072425 [FreeBSD-base] conflicts with FreeBSD-utilities-15.0.b1.20251011075131 [installed] on /usr/lib/virtual_oss/voss_null.so
  - FreeBSD-ncurses-lib-15.0.b1.20251015211959 [FreeBSD-base] conflicts with FreeBSD-ncurses-15.0.b1.20251011075131 [installed] on /lib/libncursesw.so.9
  - FreeBSD-bluetooth-lib-15.0.b1.20251013072425 [FreeBSD-base] conflicts with FreeBSD-bluetooth-15.0.b1.20251011075131 [installed] on /usr/lib/libbluetooth.so.4
  - FreeBSD-local-unbound-dev-15.0.b1.20251015211959 [FreeBSD-base] conflicts with FreeBSD-unbound-dev-15.0.b1.20251011075131 [installed] on /usr/lib/libprivateunbound.a
  - FreeBSD-local-unbound-15.0.b1.20251015211959 [FreeBSD-base] conflicts with FreeBSD-unbound-15.0.b1.20251011075131 [installed] on /etc/rc.d/local_unbound
Checking integrity... done (0 conflicting)
The following 208 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        FreeBSD-clibs-lib32: 15.0.b1.20251012072228 [FreeBSD-base]
        FreeBSD-local-unbound: 15.0.b1.20251015211959 [FreeBSD-base]
        FreeBSD-local-unbound-dev: 15.0.b1.20251015211959 [FreeBSD-base]
        FreeBSD-ncurses-lib: 15.0.b1.20251015211959 [FreeBSD-base]

Installed packages to be UPGRADED:
        FreeBSD-acct: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]
        FreeBSD-acpi: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]
        FreeBSD-apm: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]
        (...)
        FreeBSD-zfs-dev: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]
        FreeBSD-zfs-lib: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]
        FreeBSD-zoneinfo: 15.0.b1.20251011075131 -> 15.0.b1.20251012072228 [FreeBSD-base]

Installed packages to be REMOVED:
        FreeBSD-unbound: 15.0.b1.20251011075131
        FreeBSD-unbound-dev: 15.0.b1.20251011075131

Number of packages to be removed: 2
Number of packages to be installed: 4
Number of packages to be upgraded: 202

The process will require 4 MiB more space.

Proceed with this action? [y/N]: y
[1/281] Upgrading FreeBSD-kernel-generic from 15.0.b1.20251011075131 to 15.0.b2.20251017190138...
[1/281] Extracting FreeBSD-kernel-generic-15.0.b2.20251017190138: 100%
[2/281] Deinstalling FreeBSD-set-base-15.0.b1.20251011075131...
(...)
[279/281] Extracting FreeBSD-zfs-dev-15.0.b1.20251012072228: 100%
[280/281] Installing FreeBSD-set-devel-15.0.b1.20251015211959...
[281/281] Installing FreeBSD-set-base-15.0.b1.20251012072228...

root@pkgbase:~ # freebsd-version -k
15.0-BETA2

root@pkgbase:~ # freebsd-version -u
15.0-BETA2

root@pkgbase:~ # freebsd-version -r
15.0-BETA1

root@pkgbase:~ # uname -prism
FreeBSD 15.0-BETA1 amd64 amd64 GENERIC

We may reboot now.

In both cases – for some reason – the FreeBSD-clibs-lib32 packages was pulled in as dependency … but You can delete it afterwards – it does not have any deps or reqs.

root@pkgbase:~ # pkg info | grep lib32
FreeBSD-clibs-lib32-15.0.b1.20251012072228 Core runtime libraries (32-bit libraries)

root@pkgbase:~ # pkg delete FreeBSD-clibs-lib32-15.0.b1.20251012072228
Cannot solve problem using SAT solver, trying another plan
(...)
Cannot solve problem using SAT solver, trying another plan
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
        FreeBSD-clibs-lib32: 15.0.b1.20251012072228

Number of packages to be removed: 1

The operation will free 4 MiB.

Proceed with deinstalling packages? [y/N]: y
[1/1] Deinstalling FreeBSD-clibs-lib32-15.0.b1.20251012072228...
[1/1] Deleting files for FreeBSD-clibs-lib32-15.0.b1.20251012072228: 100%

Vital

Some packages will be marked as vital to prevent pkg delete -a from working and making damage – but vital concept does NOT protect against pkg delete -af since the understanding is that force -f flag specifically means “I know what I am doing – remove the packages at all costs.” … You have been warned.

PKGBASE Jails

The bsdinstall(8) installer has been updated with pkgbase --jail option that allows to populate a directory as FreeBSD Jail.

This is the recipe.

root@pkgbase:~ # mkdir -pv /jail/NEW
/jail
/jail/NEW

root@pkgbase:~ # export BSDINSTALL_CHROOT=/jail/NEW

root@pkgbase:~ # env TERM=xterm bsdinstall pkgbase --jail

Next select needed sets and proceed.

After installation there are 178 pkg(8) packages installed.

root@pkgbase:~ # pkg -r /jail/NEW info | wc -l
     178

root@pkgbase:~ # pkg -r /jail/NEW info -as | sort -k 2 -h | column -t | tail -10
FreeBSD-libmagic-15.snap20251011015136              11.8MiB
FreeBSD-utilities-dev-15.snap20251017141652         17.1MiB
FreeBSD-lldb-15.snap20251011015136                  18.4MiB
FreeBSD-zfs-dev-15.snap20251011015136               22.1MiB
FreeBSD-locales-15.snap20251011015136               24.3MiB
FreeBSD-openssl-dev-15.snap20251011015136           32.8MiB
FreeBSD-utilities-15.snap20251017141652             49.3MiB
FreeBSD-clibs-dev-15.snap20251018120849             59.2MiB
FreeBSD-clang-dev-15.snap20251011015136             70.4MiB
FreeBSD-clang-15.snap20251015183322                 149MiB

root@pkgbase:~ # du -smA /jail/NEW
804     /jail/NEW

Keep in mind that the pkg(8) is still not bootstrapped – we can do that now.

root@pkgbase:~ # chroot /jail/NEW/

[jail] root@pkgbase:/ # echo nameserver 9.9.9.9 > /etc/resolv.conf

[jail] root@pkgbase:/ # mount -t devfs devfs /dev

[jail] root@pkgbase:/ # pkg info
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64/latest, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
Updating FreeBSD-ports repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   10 MiB   2.6MB/s    00:04    
Processing entries: 100%
FreeBSD-ports repository update completed. 36389 packages processed.
Updating FreeBSD-ports-kmods repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01    
Fetching data.pkg: 100%   29 KiB  29.3kB/s    00:01    
Processing entries: 100%
FreeBSD-ports-kmods repository update completed. 200 packages processed.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.

… as you not need CLANG compiler in the Jail you may as well remove that part – same as many others – this is where PKGBASE helps.

[jail] root@pkgbase:/ # pkg delete -f FreeBSD-clang-15.snap20251015183322 FreeBSD-clang-dev-15.snap20251011015136
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 2 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
        FreeBSD-clang: 15.snap20251015183322
        FreeBSD-clang-dev: 15.snap20251011015136

Number of packages to be removed: 2

The operation will free 220 MiB.

Proceed with deinstalling packages? [y/N]: y
[1/2] Deinstalling FreeBSD-clang-dev-15.snap20251011015136...
[1/2] Deleting files for FreeBSD-clang-dev-15.snap20251011015136: 100%
[2/2] Deinstalling FreeBSD-clang-15.snap20251015183322...
[2/2] Deleting files for FreeBSD-clang-15.snap20251015183322: 100%

[jail] root@pkgbase:/ # du -smA .
704     .

100 MB less space.

Additional Independent Rescue

Often in a serious problem the FreeBSD Rescue subsystem located at /rescue is the last resort help … but it will also be removed – as any other package.

The good news is that nothing prevents You from creating your own even more secure /RESCUE that will be entirely independent from pkg(8) command operations.

Do not just copy /rescue dir with cp(1) command as it will take almost 3 GB (1.5 GB after ZFS compression) with such operation 🙂

root@pkgbase:/ # cp -ap /rescue /RESCUE
  
root@pkgbase:/ # du -sm /rescue /RESCUE
10      /rescue
1357    /RESCUE
  
root@pkgbase:/ # du -smA /rescue /RESCUE
20      /rescue
2855    /RESCUE

Here is how to create it in intelligent way – so it will only took the same small amount of space as original /rescue – with ln(1) hardlinks.

root@pkgbase:/ # mkdir /RESCUE

root@pkgbase:/ # cp /rescue/bectl /RESCUE/bectl

root@pkgbase:/ # cp /usr/local/sbin/pkg-static /RESCUE

root@pkgbase:/ # tar -cf /RESCUE/boot.tar /boot
tar: Removing leading '/' from member names
root@pkgbase:/ # ls -1 /rescue | while read I; do ln /RESCUE/bectl /RESCUE/"${I}"; done
ln: /RESCUE/bectl and /RESCUE/bectl are the same directory entry

root@pkgbase:/ # du -sm /rescue /RESCUE
11      /rescue
99      /RESCUE

root@pkgbase:/ # ls /rescue | wc -l
     148

root@pkgbase:/ # ls /RESCUE | wc -l
     152

root@pkgbase:/ # /RESCUE/bectl list
BE      Active Mountpoint Space Created
backup  -      -          393M  2025-10-18 23:35
default NR     /          1.42G 2025-10-18 16:31
new     -      -          8K    2025-10-20 12:46

You now have additional protection in case of serious emergency.

Our new /RESCUE is even better then the original on as we also have pkg-static(8) there … and a copy of /boot directory – so even if you wipe all FreeBSD system with pkg delete -fay you can restore the /boot and reboot into other ZFS Boot Environment 🙂

The rescue procedure looks like that one below.

root@pkgbase:~ # /RESCUE/tar -C / -xf /RESCUE/boot.tar
tar: Failed to set default locale
boot/efi/: Can't restore time: Invalid argument
tar: Error exit delayed from previous errors.

root@pkgbase:~ # /RESCUE/reboot

After reboot the FreeBSD loader(8) will welcome you with possibility to select different ZFS Boot Environment.

In the Works …

As final PKGBASE world had not yet settled – expect additions and updates to this article as soon as new info gets to me.

… and feel free to share your findings and hints for living in the PKGBASE world.

UPDATE 1 – Safely Remove All Third Party Packages

After evaluating possible ways I currently use this solution below as a replacement for pkg delete -yaf from before PKGBASE times.

First – the pkg query '%o %R' prints ORIGIN and REPOSITORY as list.

The column(1) command used only to make it more readable/aligned.

pkgbase # pkg query '%o %R' | column -t
(...)
base/FreeBSD-zlib                  FreeBSD-base
base/FreeBSD-zlib-dev              FreeBSD-base
base/FreeBSD-zoneinfo              FreeBSD-base
sysutils/beadm                     FreeBSD-ports
sysutils/lsblk                     FreeBSD-ports
ports-mgmt/pkg                     FreeBSD-base

So we can check the 2nd column using awk(1) for FreeBSD-ports value … and print only the ORIGIN.

pkgbase # pkg query '%o %R' | awk '$2=="FreeBSD-ports" {print $1}'
sysutils/beadm
sysutils/lsblk

Notice that since PKGBASE the pkg(8) is now registered as PKGBASE package.

Now – to remove all third party packages we can do this below.

pkgbase # pkg query '%o %R' | awk '$2=="FreeBSD-ports" {print $1}' | xargs -n1 echo pkg delete -f -y
pkg delete sysutils/beadm
pkg delete sysutils/lsblk

Now – to REALLY remove these packages first remove the echo safety switch from the command above to make it work – because right now it only prints instructions that will remove packages.

EOF
Top

Scripts Stats

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

I have grown many scripts in my UNIX life – close to 500 that I still use daily and even more then 700 in my 18 years time at various jobs … and also often for personal daily reasons.

% find ~/scripts -maxdepth 1 | wc -l
     492

% find ~/scripts -type f | wc -l
     702

All (or most) of these scripts are available on my https://github.com/vermaden/scripts GitHub page.

Feel free to ask me about ones I did not added there.

… and while having all of them – I was curious … how often I use them.

Are they still useful? Should I delete them? Phase out? Archive?

There was time that You probably seen the following code part in some of my scripts.

echo '1' 2> /dev/null >> ~/scripts/stats/${0##*/}

It is/was for stats.

I have created a ~/scripts/stats directory in which I stored stats for how often I executed my scripts … and gathered these stats for 3 years.

The start date of the experiment was on 2020/10/17 day.

To be honest – the code snipped should have looked like that one below – to make sure its possible to store the stats and the dir exists 🙂

mkdir -p                 ~/scripts/stats || exit 0
echo '1' 2> /dev/null >> ~/scripts/stats/${0##*/}

The complete stats are below – but be advised that they contain various scripts I run using cron(8) or that they are part of my 5 minutes or less Dzen2 information bar.

5800969  TOTAL
1074619  desktop-kill-shit.sh
 506653  __conky_if_dns.sh
 506485  __conky_if_ping.sh
 506466  __conky_if_gw.sh
 506357  dzen2-update.sh
 499821  __conky_if_ip.sh
 493590  __conky_battery_separate.sh
 484046  desktop-battery-warning.sh
 363833  __conky_if_xfer.sh
 207484  rsync-delete.sh
 132050  acpi-thinkpad-fan.sh
 155405  random-wallpaper.sh
  94059  rsync-delete-before.sh
  42239  xdotool.sh
  41625  to-ascii.sh
  37234  random.sh
  25154  xterm.sh
  20593  random-wallpaper-always.sh
  12811  __conky_battery.sh
  11162  todo.sh
  11135  cal.sh
   8634  network.sh
   6303  mpv.sh
   6249  ff.mp3.320.sh
   6017  mupdf.sh
   4771  see.sh
   3565  shot.sh
   2977  __openbox_lock.sh
   2034  urxvt.sh
   2024  aero-snap.sh
   1702  terminal.sh
   1236  desktop-debug.sh
   1082  photo-unsharp-mask-JPG.sh
   1073  dzen2-fifo.sh
   1073  __openbox_restart_dzen2.sh
   1072  __openbox_restart_tint2.sh
    788  thunar-mass-rename.sh
    741  __openbox_freebsd_sound.sh
    724  __openbox_lock_zzz.sh
    694  generate-links.sh
    683  __openbox_cpufreq_dynamic.sh
    682  __openbox_delete_wallpaper.sh
    655  __openbox_restart_conky.sh
    637  backup-cfg.sh
    620  photo-movie-audio-ac3.sh
    508  __openbox_virtualbox.sh
    503  to-lower.sh
    483  desktop-pause.sh
    468  __openbox_clean_tmp.sh
    460  ncdu.sh
    434  __openbox_cpufreq.sh
    420  firefox.sh
    384  rsync-delete-checksum.sh
    377  nfs.sh
    365  __openbox_edit_screenshot.sh
    342  games-sensible-world-of-soccer-SFX.sh
    322  sumatra.sh
    291  mdconfig.sh
    286  __openbox_dmenu.sh
    285  __random.sh
    284  mouse-move.sh
    277  photo-movie-audio-copy.sh
    272  mp3-EYE3D-remove-id.sh
    231  photo-movie-audio-copy-x265.sh
    226  __openbox_restart_redshift.sh
    208  __openbox_edit_wallpaper_gimp.sh
    205  __openbox_show_screenshot.sh
    188  __openbox_current_wallpaper.sh
    184  mpv-shuffle.sh
    183  __openbox_trash_delete.sh
    182  games-baldurs-gate-II-ee-fullscreen.sh
    162  __openbox_phaseout_wallpaper.sh
    161  files-count.sh
    152  photo-movie-half-audio-copy-x265.sh
    142  rsync.sh
    142  pdf-split.sh
    135  rdesktop-mine.sh
    135  pdf-pts-scale.sh
    135  __openbox_restart_plank.sh
    118  to-ascii-all.sh
    102  battery-capacity.sh
     93  __openbox_reload_wallpaper.sh
     89  jhead.sh
     86  tile.sh
     77  photo-requality.sh
     74  photo-rename-images.sh
     68  zip.sh
     68  pdf-concat.sh
     66  colors.sh
     65  games-baldurs-gate-I-ee-fullscreen.sh
     64  thunar.sh
     57  smartwear.sh
     57  photo-rename-movies.sh
     53  tcpkill.sh
     53  fastdiff.sh
     49  __openbox_restart_compton.sh
     48  pkg-recompile.sh
     47  sensors.sh
     45  vbox-suid.sh
     45  __openbox_restart_xbindkeys.sh
     42  games-baldurs-gate-I-fullscreen.sh
     41  contacts-convert-vcf-from.sh
     40  games-baldurs-gate-I-ee-window.sh
     40  contacts-check.sh
     38  dedup.sh
     36  pdf-extract.sh
     35  photo-unsharp-mask-PNG.sh
     35  games-colin-mcrae-rally-2.0-window.sh
     32  files-big.sh
     32  epiphany.sh
     31  photo-flow.sh
     31  backup-remote-checksum.sh
     30  loop.sh
     30  firefox-clean.sh
     27  idle.sh
     24  games-icewind-dale-ee-fullscreen.sh
     23  games-sensible-world-of-soccer.sh
     23  games-heroes-of-might-and-magic-III-window.sh
     22  fc-cache.sh
     20  xnview.sh
     20  __openbox_stop_compton.sh
     19  key.sh
     18  winscp.sh
     18  games-heroes-of-might-and-magic-III-HotA-window.sh
     18  __openbox_restart_squid.sh
     17  ports-generate-distinfo.sh
     17  games-swing.sh
     16  games-diablo.sh
     15  games-baldurs-gate-II-ee-window.sh
     15  backup-sys.sh
     11  pdf-rotate-right.sh
     11  games-DOSBOX.sh
     10  nfs-clean.sh
     10  http.sh
     10  games-pro-pinball-timeshock-window.sh
     10  games-lionheart-window.sh
      9  games-heroes-of-might-and-magic-III-HotA-fullscreen.sh
      9  ff.mp3.160.sh
      9  contacts-convert-vcf-to.sh
      9  __xterm_link_open.sh
      8  sshfs.sh
      8  lsblk.sh
      8  html-strip-script.sh
      8  games-fallout-2-window.sh
      8  games-fallout-2-fullscreen.sh
      8  games-canyon-defence.sh
      8  games-all-terrain-racing.sh
      8  __openbox_monitor_off_auto.sh
      7  games-sensible-world-of-soccer-UPG.sh
      7  games-pinball-space-cadet-NATIVE.sh
      6  rdesktop_1800_1000.sh
      6  netstat.sh
      6  games-colin-mcrae-rally-2.0-fullscreen.sh
      6  games-baldurs-gate-I-window.sh
      6  __openbox_virtualbox_vmctrl.sh
      6  __openbox_stop_plank.sh
      6  __openbox_stats_top_cpu_RENICE.sh
      6  __openbox_stats_ps_KILLALL.sh
      5  vergallery.sh
      5  rsync-delete-linux.sh
      5  games-worms-armageddon-fullscreen.sh
      5  games-robokill-1.sh
      5  games-heroes-of-might-and-magic-II.sh
      5  freebsd-lists.sh
      5  detox.sh
      5  __openbox_virtualbox_reload.sh
      5  __openbox_stats_top_cpu_KILL.sh
      5  __conky_top.sh
      4  wm_names.sh
      4  photo-movie-half-audio-copy-x264.sh
      4  photo-movie-audio-ac3-CRF.sh
      4  gg-log.sh
      4  games-sensible-golf.sh
      4  games-robokill-2.sh
      4  games-pinball-short-circuit.sh
      4  games-lionheart-fullscreen.sh
      4  __xterm_set_title.sh
      4  __openbox_stop_dzen2.sh
      4  __openbox_stats_top_mem_RENICE.sh
      3  see-open.sh
      3  photo-FLOW.sh
      3  jail.sh
      3  jail-version.sh
      3  games-theme-hospital.sh
      3  games-supercars-international.sh
      3  games-governor-of-poker.sh
      3  games-carmageddon.sh
      3  ext-maxtor.sh
      3  __openbox_stop_tint2.sh
      3  __openbox_stats_top_mem_KILL.sh

The same list without the stuff that runs periodically in cron(8) is below.

235624  TOTAL
 42239  xdotool.sh
 41625  to-ascii.sh
 37234  random.sh
 25154  xterm.sh
 11162  todo.sh
 11135  cal.sh
  8634  network.sh
  6303  mpv.sh
  6249  ff.mp3.320.sh
  6017  mupdf.sh
  4771  see.sh
  3565  shot.sh
  2977  __openbox_lock.sh
  2034  urxvt.sh
  2024  aero-snap.sh
  1702  terminal.sh
  1236  desktop-debug.sh
  1082  photo-unsharp-mask-JPG.sh
  1073  __openbox_restart_dzen2.sh
  1072  __openbox_restart_tint2.sh
   788  thunar-mass-rename.sh
   741  __openbox_freebsd_sound.sh
   724  __openbox_lock_zzz.sh
   694  generate-links.sh
   683  __openbox_cpufreq_dynamic.sh
   682  __openbox_delete_wallpaper.sh
   655  __openbox_restart_conky.sh
   637  backup-cfg.sh
   620  photo-movie-audio-ac3.sh
   508  __openbox_virtualbox.sh
   503  to-lower.sh
   483  desktop-pause.sh
   468  __openbox_clean_tmp.sh
   460  ncdu.sh
   434  __openbox_cpufreq.sh
   420  firefox.sh
   384  rsync-delete-checksum.sh
   377  nfs.sh
   365  __openbox_edit_screenshot.sh
   342  games-sensible-world-of-soccer-SFX.sh
   322  sumatra.sh
   291  mdconfig.sh
   286  __openbox_dmenu.sh
   285  __random.sh
   284  mouse-move.sh
   277  photo-movie-audio-copy.sh
   272  mp3-EYE3D-remove-id.sh
   231  photo-movie-audio-copy-x265.sh
   226  __openbox_restart_redshift.sh
   208  __openbox_edit_wallpaper_gimp.sh
   205  __openbox_show_screenshot.sh
   188  __openbox_current_wallpaper.sh
   184  mpv-shuffle.sh
   183  __openbox_trash_delete.sh
   182  games-baldurs-gate-II-ee-fullscreen.sh
   162  __openbox_phaseout_wallpaper.sh
   161  files-count.sh
   152  photo-movie-half-audio-copy-x265.sh
   142  rsync.sh
   142  pdf-split.sh
   135  rdesktop-mine.sh
   135  pdf-pts-scale.sh
   135  __openbox_restart_plank.sh
   118  to-ascii-all.sh
   102  battery-capacity.sh
    93  __openbox_reload_wallpaper.sh
    89  jhead.sh
    86  tile.sh
    77  photo-requality.sh
    74  photo-rename-images.sh
    68  zip.sh
    68  pdf-concat.sh
    66  colors.sh
    65  games-baldurs-gate-I-ee-fullscreen.sh
    64  thunar.sh
    57  smartwear.sh
    57  photo-rename-movies.sh
    53  tcpkill.sh
    53  fastdiff.sh
    49  __openbox_restart_compton.sh
    48  pkg-recompile.sh
    47  sensors.sh
    45  vbox-suid.sh
    45  __openbox_restart_xbindkeys.sh
    42  games-baldurs-gate-I-fullscreen.sh
    41  contacts-convert-vcf-from.sh
    40  games-baldurs-gate-I-ee-window.sh
    40  contacts-check.sh
    38  dedup.sh
    36  pdf-extract.sh
    35  photo-unsharp-mask-PNG.sh
    35  games-colin-mcrae-rally-2.0-window.sh
    32  files-big.sh
    32  epiphany.sh
    31  photo-flow.sh
    31  backup-remote-checksum.sh
    30  loop.sh
    30  firefox-clean.sh
    27  idle.sh
    24  games-icewind-dale-ee-fullscreen.sh
    23  games-sensible-world-of-soccer.sh
    23  games-heroes-of-might-and-magic-III-window.sh
    22  fc-cache.sh
    20  xnview.sh
    20  __openbox_stop_compton.sh
    19  key.sh
    18  winscp.sh
    18  games-heroes-of-might-and-magic-III-HotA-window.sh
    18  __openbox_restart_squid.sh
    17  ports-generate-distinfo.sh
    17  games-swing.sh
    16  games-diablo.sh
    15  games-baldurs-gate-II-ee-window.sh
    15  backup-sys.sh
    11  pdf-rotate-right.sh
    11  games-DOSBOX.sh
    10  nfs-clean.sh
    10  http.sh
    10  games-pro-pinball-timeshock-window.sh
    10  games-lionheart-window.sh
     9  games-heroes-of-might-and-magic-III-HotA-fullscreen.sh
     9  ff.mp3.160.sh
     9  contacts-convert-vcf-to.sh
     9  __xterm_link_open.sh
     8  sshfs.sh
     8  lsblk.sh
     8  html-strip-script.sh
     8  games-fallout-2-window.sh
     8  games-fallout-2-fullscreen.sh
     8  games-canyon-defence.sh
     8  games-all-terrain-racing.sh
     8  __openbox_monitor_off_auto.sh
     7  games-sensible-world-of-soccer-UPG.sh
     7  games-pinball-space-cadet-NATIVE.sh
     6  rdesktop_1800_1000.sh
     6  netstat.sh
     6  games-colin-mcrae-rally-2.0-fullscreen.sh
     6  games-baldurs-gate-I-window.sh
     6  __openbox_virtualbox_vmctrl.sh
     6  __openbox_stop_plank.sh
     6  __openbox_stats_top_cpu_RENICE.sh
     6  __openbox_stats_ps_KILLALL.sh
     5  vergallery.sh
     5  rsync-delete-linux.sh
     5  games-worms-armageddon-fullscreen.sh
     5  games-robokill-1.sh
     5  games-heroes-of-might-and-magic-II.sh
     5  freebsd-lists.sh
     5  detox.sh
     5  __openbox_virtualbox_reload.sh
     5  __openbox_stats_top_cpu_KILL.sh
     5  __conky_top.sh
     4  wm_names.sh
     4  photo-movie-half-audio-copy-x264.sh
     4  photo-movie-audio-ac3-CRF.sh
     4  gg-log.sh
     4  games-sensible-golf.sh
     4  games-robokill-2.sh
     4  games-pinball-short-circuit.sh
     4  games-lionheart-fullscreen.sh
     4  __xterm_set_title.sh
     4  __openbox_stop_dzen2.sh
     4  __openbox_stats_top_mem_RENICE.sh
     3  see-open.sh
     3  photo-FLOW.sh
     3  jail.sh
     3  jail-version.sh
     3  games-theme-hospital.sh
     3  games-supercars-international.sh
     3  games-governor-of-poker.sh
     3  games-carmageddon.sh
     3  ext-maxtor.sh
     3  __openbox_stop_tint2.sh
     3  __openbox_stats_top_mem_KILL.sh

The experiment ended on 2023/10/17.

This is the time I removed the known code snipped from all of the scripts.

This data helped me to tweak a little more the ones that I use the most … and especially the ones that are run by my Dzen2 info bar config or in cron(8) daemon.

Some of this input also helped me to just phase out the ones that are not needed anymore … to put them into retirement.

EOF
Top

HardenedBSD December 2025 Status Report

Post by HardenedBSD via HardenedBSD »

This status report is going to be a lengthy one. Due to scheduling conflicts, I was unable to get out the November status report, this one will cover the two months November - December 2025.

A large portion of my focus has been on the infrastructure, getting a build environment for the recently-created hardened/15-stable/main branch. As discussed in a previous mailing list thread[1], the 14-STABLE build infrastructure has now been migrated to 15-STABLE. We have archived the last 14-STABLE package build, which last completed on 24 Dec 2025.

We self-host nearly the entirety of our infrastructure out of my home. We have only one leased server, from the fine folks at NetActuate (previously RootBSD). This leased server hosts our main website, the hbsd-update build artifacts, and the package repos. Our package repos, naturally, grow over time. Back when we started this, each package repo was at most 75GB in size. Now we're encroaching 135GB.

We now have a 30TB NAS in the home-based infrastructure. In order to support the growth, we will be migrating the package repo to the home infra. The package repos themselves have already been migrated. The only thing left to do is adjust the various DNS entries. I plan to do that once we have a usable 15-STABLE package repo. We will update this[2] mailing list thread when the migration has completed, DNS records and all. There will likely be a little blip in HTTPS/TLS connections as we regenerate LetsEncrypt certs. There's a delicate dance here. I plan to keep everyone informed as to when I begin and complete the process.

The 14-STABLE build server (which is now being migrated to 15-STABLE) housed two VMs:

  1. The OS installer/update build VM. This builds the artifacts published at https://installers.hardenedbsd.org/ and mirrors.
  2. The package build VM.

When we deployed that (stupendously) slow server to test its capabilities as a build server for 15-STABLE, we followed the same pattern: two separate VMs. We are going to keep the 15-STABLE OS installer/update build VM on that slow server. We're going to power off the 14-STABLE OS build VM and increase the resources to the package build VM. This means we should be able to decrease the time it takes for that server to produce a usable package repo. Naturally, this comes at a cost of a slow build time for the OS installer/updates, but that process can tolerate **a lot** of slowness. So long as it can produce its build artifacts in less than 48 hours, I'm satisfied. It's the package building (36,000+ packages) that takes the most resources.

I spent a lot of time in the ports tree over the past couple months. The focus was on fixing ports broken by the various hardening techniques we employ. The introduction of -Werror=format-security caused a large amount of fallout, which I have been addressing. While addressing those, I figured I might as well fix ports broken by the other techniques.

I'm working on enhancing libhbsdcontrol with better error handling. I'm hoping to have that work committed in early January 2026.

I'm hoping in January to spend some time on hbsdfw. The VM I've been using to build hbsdfw has been panicking when the Poudriere build finishes when building the hbsdfw packages. In Q1 2026, I plan to migrate hbsdfw from HardenedBSD 14-STABLE to 16-CURRENT. Following the hardened/current/master src branch will lighten my load in maintaining this little hobby subproject.

I need to file a bug report upstream in FreeBSD/OpenZFS to track this kernel panic. The panic happens when something during the build checks whether PaX PAGEEXEC is enabled through looking up a filesystem extended attribute. OpenZFS recently changed how filesystem extended attributes work, so it's possible we're hitting a unique edge case.

In January, I'm going to get two lab environments set up:

  1. Internal Reticulum nodes to test the Reticulum protocol and its potential for use with our censorship- and surveillance-resistant mesh network R&D.
  2. Internal Radicle nodes to start concerted testing to eventually replace GitLab with Radicle.

I feel somewhat down for not making more progress this year on the censorship- and surveillance-resistant networks. I'm hoping to place more emphasis on this in 2026.

In src:

  1. Always build elftc-nm and elft-ar
  2. TPE: Ensure user-owned vnodes are unwritable
  3. ASLR: Use VMFS_NO_SPACE to map the stack
  4. Add various C/C++ hardening flags:
    1. -fno-delete-null-pointer-checks
    2. -Werror=format-security
  5. Unlock the sound mutex on error
  6. Fix branch detection in release
  7. Disable SafeStack for the Unbound daemon
  8. Some pkgbase-related work

In ports (this is gonna be a long list (our longest to date)):

  1. Disable LINUX for x11/nvidia-kmod
  2. ftp/curl: Fixup .onion patch
  3. Add "general compilation hardening" USES
  4. Delete unneeded patch for databases/redis
  5. Fix archivers/zip
  6. Disable hardcflags for devel/m4
  7. Disable hardcflags for lang/gcc13
  8. Disable HARDCFLAGS for devel/t1lib
  9. Fix HARDCFLAGS errors for devel/ctags
  10. Disable HARDCFLAGS for archivers/unzip
  11. Fix HARDCFLAGS for net-mgmt/libsmi
  12. Disable HARDCFLAGS for x11-toolkits/open-motif
  13. Disable HARDCFLAGS for devel/expect
  14. Fix the devel/ivykis port
  15. Fix HARDCFLAGS for multimedia/webcamd
  16. Disable HARDCFLAGS for lang/gcc12
  17. Disable HardenedBSD features for lang/gcc14
  18. Disable HardenedBSD features for lang/gcc15
  19. Disable HardenedBSD features for lang/gcc16-devel
  20. Fix HARDCFLAGS for multimedia/smpeg
  21. Disable HARDCFLAGS for devel/elfutils
  22. Fix HARDCFLAGS for converters/recode
  23. Disable fortifysource for graphics/netpbm
  24. Fix hardcflags for devel/fortytwo-encore
  25. Fix HARDCFLAGS for graphics/libvisual04
  26. Disable HARDCFLAGS for devel/kBuild
  27. Fix HARDCFLAGS for devel/libbegemot
  28. Fix HARDCFLAGS for games/pmars-sdl
  29. Disable FORTIFYSOURCE for security/signify
  30. Disable HARDCFLAGS for mail/mailutils
  31. Fix HARDCFLAGS for devel/ta-lib
  32. Fix HARDCFLAGS for math/spooles
  33. Fix HARDCFLAGS for textproc/wv
  34. Fix HARDCFLAGS for databases/sqlite2
  35. Disable HARDCFLAGS for graphics/lensfun
  36. Fix HARDCFLAGS for devel/rlwrap
  37. Disable fortifysource for mail/opensmtpd
  38. Fix HARDCFLAGS for x11-toolkits/unique
  39. Fix HARDCFLAGS for devel/efivar
  40. Fix HARDCFLAGS for lang/f2c
  41. Fix HARDCFLAGS for textproc/scim-table-imengine
  42. Disable FORTIFYSOURCE and HARDCFLAGS for sysutils/fwupd-efi
  43. Fix HARDCFLAGS for games/libmt_client
  44. Disable HARDCFLAGS for games/gnugo
  45. Fix HARDCFLAGS for comms/rxtx
  46. Disable PIE and RELRO for databases/redis
  47. Fix build for devel/omniORB
  48. Fix build of security/rubygem-bcrypt_pbkdf
  49. Fix HARDCFLAGS for math/grace
  50. Fix HARDCFLAGS for audio/libbs2b
  51. Disable HARDCFLAGS for graphics/plotutils
  52. Fix HARDCFLAGS for emulators/libretro-reicast
  53. Add -Wformat for HARDCFLAGS
  54. Disable HARDCFLAGS for graphics/gracula
  55. Fix HARDCFLAGS for mail/spmfilter
  56. Add cheat support in games/ioquake3
  57. Fix HARDCFLAGS for print/catdvi
  58. Fix HARDCFLAGS for graphics/seom
  59. Fix HARDCFLAGS for deskutils/presage
  60. Fix HARDCFLAGS for graphics/alpng
  61. Enable SLH for games/ioquake3
  62. Fix -Werror=format-security bug in games/ioquake3
  63. Fix HARDCFLAGS for x11-toolkits/fox16
  64. Disable HARDCFLAGS for graphics/glslang
  65. Re-enable PIE and RELRO for databases/redis
  66. Fix HARDCFLAGS for converters/uudeview
  67. Fix HARDCFLAGS for textproc/gdome2
  68. Disable FORTIFYSOURCE for misc/mbuffer
  69. Disable HARDCFLAGS for archivers/unarj
  70. Disable FORTIFYSOURCE for misc/amanda-{client,server}
  71. Disable FORTIFYSOURCE for net/dante
  72. Fix HARDCFLAGS for archivers/sharutils
  73. Fix HARDCFLAGS for lang/squeak
  74. Disable FORTIFYSOURCE for devel/socket_wrapper
  75. Fix HARDCFLAGS for net/pvm
  76. Fix HARDCFLAGS for audio/snack
  77. Fix HARDCFLAGS for textproc/sgmlformat
  78. Fix HARDCFLAGS for cad/iverilog
  79. Fix HARDCFLAGS for sysutils/genisoimage
  80. Disable HARDCFLAGS for games/libretro-boom3
  81. Fix HARDCFLAGS for math/testu01
  82. Disable FORTIFYSOURCE for devel/pcc-libs
  83. Disable PIE for security/cryptlib
  84. Fix HARDCFLAGS for mail/addresses-goodies
  85. Fix build of devel/ivykis on 14-stable
  86. Disable HARDCFLAGS for security/pgpin
  87. (0x1eef) Fix grub2-bhyve build error
  88. Disable HARDCFLAGS for devel/cunit
  89. Disable FORTIFYSOURCE for editors/dte
  90. Disable FORTIFYSOURCE for mail/akpop3d
  91. Disable HARDCFLAGS for emulators/x48
  92. Fix HARDCFLAGS for net/osrtspproxy
  93. Fix HARDCFLAGS for mail/qmailmrtg7
  94. Fix HARDCFLAGS for print/transfig
  95. Disable PIE for graphics/nsxiv
  96. Disable FORTIFYSOURCE for devel/uid_wrapper
  97. Disable HARDCFLAGS for devel/cweb
  98. Fix FORTIFYSOURCE for multimedia/ffmpeg
  99. Fix build of lang/gcc14
  100. Fix FORTIFYSOURCE for devel/tex-libtexluajit
  101. Disable FORTIFYSOURCE and HARDCFLAGS for security/barnyard2
  102. Fix build of lang/gcc12
  103. Fix build of databases/arrow

[1]: https://groups.google.com/a/hardenedbsd.org/g/users/c/51IARO8noYo/m/asRq...
[2]: https://groups.google.com/a/hardenedbsd.org/g/users/c/G6HbsE8DA5w/m/I4ou...

Top

Lime wash – over field stone walls already covered in latex and/or lead-base paint

Post by Dan Langille via Dan Langille's Other Diary »

I have a plan. To paint the basement walls, without paint, but instead a lime-based wash.

photos: https://www.flickr.com/photos/dlangille/albums/72177720331055034/
The basement is 46x15ft with 9ft tall walls. or 2 * (46 + 15) * 9 = 1098 sq ft.

https://www.limeworks.us/calculator/calc_limepaint.php# says I need 15 gallons of Ecologic™ LimeWash – $125.00 / 5 gallon. or $375

Lancaster Limeworks makes

* https://lancasterlimeworks.com/product/plastrum-limewash/
* https://lancasterlimeworks.com/product/natural-hydraulic-lime-2/

Limeworks makes:

* https://www.limeworks.us/product/ecologic-whitewash-platinum/
* https://www.limeworks.us/product/st-astier-naturally-hydraulic-lime/

Let’s see how much NHL 2.0 costs in comparison: estimates start at 75 to 150 square feet per gallon. 1098 sq ft means 14 gallons.

One recipe:

“To make limewash with Natural Hydraulic Lime (NHL) 2.0, mix NHL 2 powder with water to a thin, creamy consistency (around 1 part lime to 4-6 parts water),”

1 gallon of the Limeworks NHL is $27.00. Mixing 4×1 sounds like by volume, not weight. It should make 4 gallons. 16 bags x $27 is $432

They also sell 55 pound bags for $72 – which should make 26 gallons – way more than I need.

Lancaster Limeworks sells a 20K bag (44lbs) for $48 which should make 20 gallons.

More to follow there. These notes will be updated as more information comes in.

Posted to the reddit thread: https://www.limeworks.us/wp-content/uploads/2019/11/whitewash-recipe.pdf

Top

2025: Software Development and Infrastructure Support.

Post by FreeBSD Foundation via FreeBSD Foundation »

As we look back on 2025, it is clear that this has been a year of meaningful progress for the FreeBSD Project and the FreeBSD Foundation. We advanced key development initiatives, strengthened core infrastructure, improved accessibility, and continued supporting enterprise-scale use. 

This work was made possible through the generosity of our donors and the dedication of contributors, partners, and staff. With 62% of our annual budget invested directly in software development, the Foundation remained focused on delivering sustainable, high-impact improvements across the ecosystem. The sections below highlight the technical achievements that shaped 2025.

 

 

Major Software Development Highlights:

In 2025, hardware enablement remained one of the most visible and high-impact areas of Foundation-funded work. Significant progress was made in wireless networking, graphics, audio, and power management, areas that directly improve the day-to-day experience for users running FreeBSD on modern laptops and workstations. 

Foundation-supported work this year included:

These improvements meaningfully reduce barriers to laptop use while strengthening hardware support across the broader FreeBSD ecosystem.

Infrastructure Modernization – commissioned by the Sovereign Tech Agency

In 2024 The Sovereign Tech Agency commissioned an ambitious body of work to strengthen and modernize the infrastructure that FreeBSD contributors depend on. 

The program of work totalling €686,400 was managed by the FreeBSD Foundation and has run from August 2024 to December 2025. The main goals of the program were to accelerate planned work to deliver zero trust builds, SBOM and security tooling, and improve developer experience.

As the project nears completion, some of the key work delivered includes:

  • Zero Trust Builds 
  • CI/CD Automation
  • Reduce Technical Debt
  • Security Controls
  • SBOM Improvements

To read more about this work visit

Alpha-Omega Security & Supply-Chain Work

This year’s Alpha-Omega work gave us a clearer understanding of the third-party software included in the FreeBSD base system and what it will take to support those components responsibly moving forward.

We completed the OpenSSL 3.5 LTS upgrade, eliminating a major end-of-support risk and ensuring FreeBSD stays aligned with an actively maintained cryptographic library relied on by many downstream organizations.

As part of the Beach Cleaning initiative, we are significantly strengthening FreeBSD’s third-party software supply chain by developing a clearer picture of the external components we rely on and what it will take to support them responsibly over the long term.

At the same time, we built a structured, machine-readable inventory of all third-party components in the base system—an essential step in the Beach Cleaning initiative that now informs our emerging Fix / Fork / Forego decisions around maintenance and security.

Additional efforts such as SBOM generation, test suite integration, and improved tracking of upstream changes and advisories are already underway and will continue into next year. Beyond the direct improvements to FreeBSD, this project is helping shape a practical model that other open-source communities can follow as they grapple with legacy code, uneven upstream support, and growing expectations for software supply chain transparency.

Google Summer of Code 2025

GSoC continues to serve as one of our most important pipelines for onboarding new contributors and strengthening community engagement. Under the Foundation’s management, FreeBSD completed its 21st consecutive Google Summer of Code cycle this year. All 12 student projects were successfully completed, with several participants contributing directly to the 2025 status reports. Some highlights include:

GSoC continues to play a meaningful role in attracting and developing the next generation of FreeBSD contributors. Since 2022, five new FreeBSD committers have come through GSoC, and one 2017 participant went on to serve on the 12th Core Team. Learn more about FreeBSD and the Google summer of Code 2025

Code Commits Sponsored in 2025

This year, the Foundation sponsored over 2094+ commits across the src, ports, and documentation trees. These contributions reflect ongoing work that strengthens performance, security, hardware support, and developer productivity across the entire FreeBSD ecosystem.

OpenJDK Improvements

The Foundation continued sponsoring improvements to OpenJDK on FreeBSD, helping ensure that modern Java workloads are stable and well-supported across releases. 

 

FreeBSD 15: Modernization, Security, and Developer Experience

This year’s developer and vendor summits highlighted the significant advancements in FreeBSD 15. The Foundation also provided targeted funding and infrastructure support that helped advance several key initiatives contributing to FreeBSD 15. These investments focused on strengthening core systems, improving developer workflows, and supporting long-term sustainability across the Project.

  • Pkgbase Readiness
    Foundation-sponsored development and testing continued moving pkgbase toward reliable, production-ready use, improving upgrade tooling and system reproducibility.
  • Accessibility Improvements
    Foundation-funded documentation updates, including work on the Accessibility Handbook, helped make FreeBSD easier to adopt and more accessible to a broader audience.

FreeBSD 15 reflects the result of many years of community-driven effort, supported in part by the Foundation’s commitment to sustained engineering investment. For more details on the software development work in 2025 see the FreeBSD quarterly update….

Infrastructure, Compliance & Legal Readiness

Supporting modern open source requires strong governance and legal structures.
Key 2025 efforts included:

  • CRA Readiness through attestations and compliance support
  • Licensing Framework Refinements for ports and package workflows
  • Cloud Infrastructure Monitoring and Cost Management
  • Continued scaling of testing and reproducibility frameworks

These investments ensure FreeBSD remains stable, secure, and compliant in diverse enterprise and regulatory environments.

A Community Effort

This year reinforced to us that meaningful progress only happens when communities, contributors, and organizations share responsibility for sustaining the work that supports us all.

Thank you to all the developers, volunteers, testers, donors, and advocates who contributed to the Project. Your support is helping build a more modern, usable, and inclusive FreeBSD ecosystem.

Support This Work

Continued progress on laptop support, usability, and hardware enablement is made possible through community donations.

You can support ongoing improvements here:

Donate to FreeBSD Foundation | FreeBSD Foundation

The post 2025: Software Development and Infrastructure Support. first appeared on FreeBSD Foundation.

Top

Powering the Future of FreeBSD

Post by FreeBSD Foundation via FreeBSD Foundation »

Did you know that your support for the FreeBSD Foundation directly improves the FreeBSD you use every day?

FreeBSD doesn’t just happen. It’s built, tested, maintained, secured, and improved by a global community. Your donation helps make that possible in ways that directly benefit you, including:

  • A stronger, more reliable FreeBSD. More than half of the Foundation’s budget goes directly into software development projects, such as advancing laptop and desktop usability and strengthening security.
  • Smoother updates and releases you can count on. Your support keeps FreeBSD’s build, testing, and CI infrastructure running, ensuring stable upgrades and dependable releases.
  • A community that strengthens the system you rely on. Your donation helps grow contributor participation, expand education and events, and keep the ecosystem healthy for the long term.

When FreeBSD grows stronger, the systems and projects you build on top of it do too. FreeBSD is powered by its community and supported by you.

Please consider donating today! We can’t do it without you. 

Sincerely, 
Deb Goodkin
Executive Director
FreeBSD Foundation

Where the Funding Goes

Here’s a quick breakdown of where most of our funding goes. This chart illustrates the overall 2025 budget, showing that 62% of the funding is allocated to software development work. Check out the budget summary for more information on the budget breakdown. 

The post Powering the Future of FreeBSD first appeared on FreeBSD Foundation.

Top

Valuable News – 2025/12/22

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

2025 OpenZFS User and Developer Summit.
https://youtube.com/playlist?list=PLaUVvul17xSeOGarlPaZcqzsVF23PSd6w

BastilleBSD 2025 User Survey.
https://docs.google.com/forms/d/e/1FAIpQLSfC20jlNgngAki7xYGNgKCfJvQ2a0lAqpkJBotXXDVVuAteYQ/viewform

DistroWatch OpenBSD 7.8 Review.
https://distrowatch.com/weekly.php?issue=20251215#openbsd

FreeBSD Git Weekly: 2025-12-08 to 2025-12-14.
https://freebsd-git-weekly.tarsnap.net/2025-12-08.html

MidnightBSD 4.0 Brings Many Changes to Its FreeBSD 13 Derived OS.
https://phoronix.com/news/MidnightBSD-4.0

MidnightBSD 4.0 Release Notes.
https://midnightbsd.org/notes/4.0/index.html

FreeBSD Foundation – 2025 – Year of Advocacy/Community/Growth.
https://freebsdfoundation.org/blog/2025-a-year-of-advocacy-community-and-growth/

Installing Every Window Manager on FreeBSD.
https://youtube.com/watch?v=q4PNSeNXEww

Adopting FreeBSD maomao Port.
https://evilham.eu/es/blog/2025-FreeBSD-adopting-a-port-maomaowm/

FreeBSD Desktop Installer Preview – Is It Any Good?
https://youtube.com/watch?v=rKsdrGr1iBs

OPNsense Mullvad WireGuard Setup. Complete Step-by-Step Guide.
https://youtube.com/watch?v=toB_9F-VTVo

12 Days of ZFS: Tips/Tricks/Treats.
https://youtube.com/watch?v=TlhRBrp9VDk

NetBSD Most Secure OS Ever.
https://drsaracco.wordpress.com/2012/12/31/netbsd-most-secure-os-ever/

How to Install and Configure FreeBSD (2027 Edition).
https://youtube.com/watch?v=TmFl7QTvWPs

New pfSense Bug Could Let Hackers In! Do This Now.
https://youtube.com/watch?v=Vqy6o_tY4Qs

OpenBSD Stories – Scariest Boot Loader Code.
http://miod.online.fr/software/openbsd/stories/boot_hppa.html

FreeBSD Closes Laptop Gap: Year One Project Update.
https://freebsdfoundation.org/blog/freebsd-closes-the-laptop-gap-year-one-project-update/

2025 Brought Transformative Changes for FreeBSD on Laptops.
https://phoronix.com/news/FreeBSD-Laptops-2025

I Have Used Linux for Decades but I Would Switch to FreeBSD for This One Feature.
https://zdnet.com/article/freebsd-will-challenge-your-skills-and-make-you-learn-along-the-way/

OPNsense 25.7.10 Released and Fixes Critical FreeBSD Flaw.
https://youtube.com/watch?v=zM2L-3QhXBk

2025 FreeBSD Foundation Budget Summary.
https://freebsdfoundation.org/about-us/about-the-foundation/financials/2025-freebsd-foundation-budget-summary/

Attempt to Read UNIX V4 Tape is Underway!
https://mastodon.social/@ricci@discuss.systems/115747843746305391

UNIX V4 Tape from University of Utah.
https://archive.org/details/utah_unix_v4_raw

BSD Now 642: Look Harder
https://www.bsdnow.tv/642

Why I Am Retiring from Contributing to FreeBSD.
https://forum.opnsense.org/index.php?PHPSESSID=mc43n25k9eij03dc5eur8mrc3a&topic=50102.0

ZFS Status on NetBSD.
https://mail-index.netbsd.org/netbsd-users/2025/12/19/msg033689.html

Indiana Jones and Great Circle – FreeBSD Wine Gaming.
https://youtube.com/watch?v=zX38PoitmAQ

DragonFlyBSD VirtIO Block Driver Lands Multi Queue Support.
https://phoronix.com/news/DragonFly-VirtIO-Block-MQ

OpenZFS 2.4 Extends Linux Kernel Support to 6.18 and Supports FreeBSD 14–16.
https://linuxiac.com/openzfs-2-4-extends-linux-kernel-support-to-6-18-and-supports-freebsd-14-16/

XLibre XServer 25.1 Changes.
https://github.com/X11Libre/xserver/wiki/XLibre-XServer-25.1-Changes

Firefox in BastilleBSD FreeBSD Jail (jailfox-install) Working Guide.
https://github.com/krolingo/Firefox_Jail_GhostBSD-FreeBSD-

Hardware

Samsung to Halt SATA SSD Production Leaker Warns of Up to 18 Months of SSD Price Pressure Worse than Micron Ending Consumer RAM.
https://notebookcheck.net/Samsung-to-halt-SATA-SSD-production-leaker-warns-of-up-to-18-months-of-SSD-price-pressure-worse-than-Micron-ending-consumer-RAM.1184896.0.html

Pimped AMIGA 600 from Bad Condition to Fully Working OS 3.2 with 68030 CPU.
https://www.pimyretro.org/pimped-amiga-600/

Restoring and Customizing AMIGA 500 – Workbench 1.3/Tools/Enhancements.
https://pimyretro.org/pimped-amiga-500/

EU is Officially Dropping Its 2035 Combustion Engine Ban.
https://motor1.com/news/782012/eu-dropping-combustion-engine-ban-2035/

Intel Xeon 6980P vs. AMD EPYC 9755 128-Core Showdown.
https://phoronix.com/review/xeon-6980p-epyc-9755-2025/

New in Collection – Part 1 – Keyport 717.
https://newsletter.shifthappens.site/archive/new-in-the-collection-pt-1-keyport-717/

Life

Texas Attorney General Sues Sony/Samsung/LG/Hisense/TCL for Spying with Screenshots.
https://texasattorneygeneral.gov/news/releases/attorney-general-paxton-sues-five-major-tv-companies-including-some-ties-ccp-spying-texans

Boys (Do Not) Lag Behind: How Teachers Gender Biases Affect Student Achievement.
https://sciencedirect.com/science/article/abs/pii/S0272775718307714

Most Raw Conversation I Have Ever Had: Revisited.
https://youtube.com/watch?v=dKmuCBFPY3g

Beginning New Chapter When You Think You Are Too Old.
https://youtube.com/watch?v=6BIsbegPxwg

Surveillance: Proton Relocates Parts of Its Infrastructure from Switzerland.
https://heise.de/en/news/Surveillance-Proton-relocates-parts-of-its-infrastructure-from-Switzerland-10538664.html

Other

Software Can Be Finished.
https://rosswintle.uk/2025/10/software-can-be-finished/

2025 Worst Person in Tech.
https://twsu.forms.app/worst-person-tech-2025-round-one

Servo 0.0.3 Browser Engine Brings Better Performance and Embedding Improvements.
https://phoronix.com/news/Servo-0.0.3-Released

ZLUDA For CUDA on Non NVIDIA GPUs Enables AMD ROCm 7 Support.
https://phoronix.com/news/ZLUDA-ROCm-7

Cloudflare Error Page Generator.
https://github.com/donlon/cloudflare-error-page

No AI Here – Response to Mozilla Next Chapter from Waterfox.
https://waterfox.com/blog/no-ai-here-response-to-mozilla/

Thunderbird Expanding Microsoft Exchange and Protocol Support for 2026.
https://phoronix.com/news/Thunderbird-2026-Plans

Honest Hacker News.
https://dosaygo-studio.github.io/hn-front-page-2035/news-honest.html

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

pysembr

Post by Kristian Köhntopp via Die wunderbare Welt von Isotopp »

With the 2025.3 release, JetBrains retired the Gracie Pro Plugin, a tool that until recently provided, among other features, the option to apply structural wrapping to a file. For natural languages this feature automatically applied formatting rules similar to SemBr . SemBr helps writers by inserting line breaks at semantic boundaries rather than simply at arbitrary character counts, which keeps prose easier to edit and track changes in version control systems. SemBr is especially useful for Markdown content such as blog posts where readable plain text and consistent formatting are both important.

Losing the Gracie Pro Plugin’s functionality left a gap in my workflow. I needed a tool that can format the Markdown in blog articles in something that is roughly like SemBr. So I had to write my own: pysembr , a Python-based tool applies structural wrapping to natural language text. Using uv it is easy to install and use.

What pysembr Does

pysembr reads your plain text or Markdown files and adds line breaks at the end of each sentence. Editing such a file later will produce clear diffs, because git looks at line structures.

If a sentence is too long for a line, pysembr will try to break at “,”. If that still isn’t sufficient, it will look at language specific break words, and optionally at conjunctions.

So we offer:

  • “Semantic” line wrapping for natural language text.
  • Markdown-aware processing that respects lists, headings, and other structural elements.
  • Command-line interface for easy integration into scripts and build pipelines.
  • Lightweight and Python-native, making it easy to adapt or extend for custom use.

How to Use It

  1. Have uv installed.
  2. Clone the repository: git clone https://github.com/isotopp/pysembr.git
  3. Load the dependencies: uv sync
  4. Run it with uv: uv run pysembr --help or install it as a tool: uv tool install .

If you run it as a tool, add uv tool dir --bin to the PATH in your shell:

For example:

pysembr input.md -o output.md

This command reads input.md, applies semantic line wrapping, and writes the formatted output to output.md. More usage examples can be found in the README.md.

Top

Infrastructure Modernization – commissioned by the Sovereign Tech Agency

Post by FreeBSD Foundation via FreeBSD Foundation »

In 2024 The Sovereign Tech Agency commissioned an ambitious body of work to strengthen and modernize the infrastructure that FreeBSD contributors depend on. 

The program of work totaling €686,400 was managed by the FreeBSD Foundation and has run from August 2024 to December 2025. 

The main goals of the program were to accelerate planned work to deliver zero trust builds, SBOM and security tooling, and improve developer experience.

As the project nears completion, some of the key work delivered is:

Zero Trust Builds

FreeBSD now builds reproducibly and without root privilege.  

The FreeBSD Foundation has completed work enabling FreeBSD builds without requiring root privileges. All release artifacts (ISO images, USB memstick images, VM images, and cloud disk images) can now be created without elevated access, reducing security risks from device file creation, ownership changes, and filesystem mounting during builds.

Alongside no-root support, FreeBSD introduced reproducible builds ensuring identical sources produce identical binaries. This involved normalizing timestamps, stabilizing file ordering, and creating consistent build environments. These improvements strengthen supply chain integrity, enable unprivileged container-based CI systems, and allow contributors to build complete releases locally, making FreeBSD management faster, more secure, and transparent.

CI/CD Automation

A package of changes to CI tooling will improve the development workflow by enabling developers to test code changes before merging and extending automated testing to the Ports tree. This will make debugging easier by providing better test metadata, automated code analysis, and notifications to code owners when issues arise.

Tooling will soon* be available which:

  • Extends CI to the Ports tree
  • Gives developers the ability to run CI (locally or in the cloud) on their proposed source code changes before they are merged into main. 
  • Collects test environment metadata to help with reproducing bugs.
  • Supports running CI in 3rd party services.
  • Provides automated code analysis as part of CI.
  • Notifies code owners when a test fails and provides details about related code changes.

Also, part of the project:

  • Automated updates to keep CI tooling up-to-date.
  • A CI threat model to support future security planning.

* This work has been sitting behind 15.0 release and will take a little longer to deliver.

Reduce Technical Debt

The Foundation worked with the Source Manager team to specify and create an analytics dashboard to gather insights from across the different tools containing information about bugs and technical debt. This was combined with a focus in the community on “bugbusting” sessions, some Bugzilla upgrades, and related new tooling to apply patches automatically. The changes have meant that there has been a sustained improvement in bug management. Over the last year the rate of closing bugs has been higher than the rate of bugs being raised.

Security Controls

Changes have been made to support FreeBSD’s adoption of the emerging OSV (Open Source Vulnerability) format for its vulnerability data.  This standardization makes it easier for downstream users to access and process security information using existing ecosystem tools, while also simplifying imports of vulnerability data from FreeBSD’s third-party components.

An OSV database for FreeBSD has been created, and OSV parsing capability has been added to pkg. Conversion tools are also available to transform existing VuXML data to OSV, with CI to automatically validate the output. pkg audit can also now handle OSV data. 

FreeBSD was also added to the upstream OSV schema to allow 3rd-party tooling to be updated to correctly handle FreeBSD OSV data.

SBOM Improvements

Foundational tooling to generate SBOMs for FreeBSD has been created by consolidating scattered provenance data into unified reports. The Ports tree implementation is mature and ready for review, while Base system SBOM generation remains in technical preview due to its complex build system. A follow-on project in early 2026 will build on this groundwork to deliver production-ready SBOM capabilities across the entire FreeBSD stack.

The post Infrastructure Modernization – commissioned by the Sovereign Tech Agency first appeared on FreeBSD Foundation.

Top

Dismantling Defenses: Trump 2.0 Cyber Year in Review

Post by Brian Krebs via Krebs on Security »

The Trump administration has pursued a staggering range of policy pivots this past year that threaten to weaken the nation’s ability and willingness to address a broad spectrum of technology challenges, from cybersecurity and privacy to countering disinformation, fraud and corruption. These shifts, along with the president’s efforts to restrict free speech and freedom of the press, have come at such a rapid clip that many readers probably aren’t even aware of them all.

FREE SPEECH

President Trump has repeatedly claimed that a primary reason he lost the 2020 election was that social media and Big Tech companies had conspired to silence conservative voices and stifle free speech. Naturally, the president’s impulse in his second term has been to use the levers of the federal government in an effort to limit the speech of everyday Americans, as well as foreigners wishing to visit the United States.

In September, Donald Trump signed a national security directive known as NSPM-7, which directs federal law enforcement officers and intelligence analysts to target “anti-American” activity, including any “tax crimes” involving extremist groups who defrauded the IRS. According to extensive reporting by journalist Ken Klippenstein, the focus of the order is on those expressing “opposition to law and immigration enforcement; extreme views in favor of mass migration and open borders; adherence to radical gender ideology,” as well as “anti-Americanism,” “anti-capitalism,” and “anti-Christianity.”

Earlier this month, Attorney General Pam Bondi issued a memo advising the FBI to compile a list of Americans whose activities “may constitute domestic terrorism.” Bondi also ordered the FBI to establish a “cash reward system” to encourage the public to report suspected domestic terrorist activity. The memo states that domestic terrorism could include “opposition to law and immigration enforcement” or support for “radical gender ideology.”

The Trump administration also is planning to impose social media restrictions on tourists as the president continues to ramp up travel restrictions for foreign visitors. According to a notice from U.S. Customs and Border Protection (CBP), tourists — including those from Britain, Australia, France, and Japan — will soon be required to provide five years of their social media history.

The CBP said it will also collect “several high value data fields,” including applicants’ email addresses from the past 10 years, their telephone numbers used in the past five years, and names and details of family members. Wired reported in October that the US CBP executed more device searches at the border in the first three months of the year than any other previous quarter.

The new requirements from CBP add meat to the bones of Executive Order 14161, which in the name of combating “foreign terrorist and public safety threats” granted broad new authority that civil rights groups warn could enable a renewed travel ban and expanded visa denials or deportations based on perceived ideology. Critics alleged the order’s vague language around “public safety threats,” creates latitude for targeting individuals based on political views, national origin, or religion. At least 35 nations are now under some form of U.S. travel restrictions.

CRIME AND CORRUPTION

In February, Trump ordered executive branch agencies to stop enforcing the U.S. Foreign Corrupt Practices Act, which froze foreign bribery investigations, and even allows for “remedial actions” of past enforcement actions deemed “inappropriate.”

The White House also disbanded the Kleptocracy Asset Recovery Initiative and KleptoCapture Task Force — units which proved their value in corruption cases and in seizing the assets of sanctioned Russian oligarchs — and diverted resources away from investigating white-collar crime.

Also in February, Attorney General Pam Bondi dissolved the FBI’s Foreign Influence Task Force, an entity created during Trump’s first term designed to counter the influence of foreign governments on American politics.

In March 2025, Reuters reported that several U.S. national security agencies had halted work on a coordinated effort to counter Russian sabotage, disinformation and cyberattacks. Former President Joe Biden had ordered his national security team to establish working groups to monitor the issue amid warnings from U.S. intelligence that Russia was escalating a shadow war against Western nations.

In a test of prosecutorial independence, Trump’s Justice Department ordered prosecutors to drop the corruption case against New York Mayor Eric Adams. The fallout was immediate: Multiple senior officials resigned in protest, the case was reassigned, and chaos engulfed the Southern District of New York (SDNY) – historically one of the nation’s most aggressive offices for pursuing public corruption, white-collar crime, and cybercrime cases.

When it comes to cryptocurrency, the administration has shifted regulators at the U.S. Securities and Exchange Commission (SEC) away from enforcement to cheerleading an industry that has consistently been plagued by scams, fraud and rug-pulls. The SEC in 2025 systematically retreated from enforcement against cryptocurrency operators, dropping major cases against Coinbase, Binance, and others.

Perhaps the most troubling example involves Justin Sun, the Chinese-born founder of crypto currency company Tron. In 2023, the SEC charged Sun with fraud and market manipulation. Sun subsequently invested $75 million in the Trump family’s World Liberty Financial (WLF) tokens, became the top holder of the $TRUMP memecoin, and secured a seat at an exclusive dinner with the president.

In late February 2025, the SEC dropped its lawsuit. Sun promptly took Tron public through a reverse merger arranged by Dominari Securities, a firm with Trump family ties. Democratic lawmakers have urged the SEC to investigate what they call “concerning ties to President Trump and his family” as potential conflicts of interest and foreign influence.

In October, President Trump pardoned Changpeng Zhao, the founder of the world’s largest cryptocurrency exchange Binance. In 2023, Zhao and his company pled guilty to failing to prevent money laundering on the platform. Binance paid a $4 billion fine, and Zhao served a four-month sentence. As CBS News observed last month, shortly after Zhao’s pardon application, he was at the center of a blockbuster deal that put the Trump’s family’s WLF on the map.

“Zhao is a citizen of the United Arab Emirates in the Persian Gulf and in May, an Emirati fund put $2 billion in Zhao’s Binance,” 60 Minutes reported. “Of all the currencies in the world, the deal was done in World Liberty crypto.”

SEC Chairman Paul Atkins has made the agency’s new posture towards crypto explicit, stating “most crypto tokens are not securities.” At the same time, President Trump has directed the Department of Labor and the SEC to expand 401(k) access to private equity and crypto — assets that regulators have historically restricted for retail investors due to high risk, fees, opacity, and illiquidity. The executive order explicitly prioritizes “curbing ERISA litigation,” and reducing accountability for fiduciaries while shifting risk onto ordinary workers’ retirement savings.

At the White House’s behest, the U.S. Treasury in March suspended the Corporate Transparency Act, a law that required companies to reveal their real owners. Finance experts warned the suspension would bring back shell companies and “open the flood gates of dirty money” through the US, such as funds from drug gangs, human traffickers, and fraud groups.

Trump’s clemency decisions have created a pattern of freed criminals committing new offenses, including Jonathan Braun, whose sentence for drug trafficking was commuted during Trump’s first term, was found guilty in 2025 of violating supervised release and faces new charges.

Eliyahu Weinstein, who received a commutation in January 2021 for running a Ponzi scheme, was sentenced in November 2025 to 37 years for running a new Ponzi scheme. The administration has also granted clemency to a growing list of white-collar criminals: David Gentile, a private equity executive sentenced to seven years for securities and wire fraud (functionally a ponzi-like scheme), and Trevor Milton, the Nikola founder sentenced to four years for defrauding investors over electric vehicle technology. The message: Financial crimes against ordinary investors are no big deal.

At least 10 of the January 6 insurrectionists pardoned by President Trump have already been rearrested, charged or sentenced for other crimes, including plotting the murder of FBI agents, child sexual assault, possession of child sexual abuse material and reckless homicide while driving drunk.

The administration also imposed sanctions against the International Criminal Court (ICC). On February 6, 2025, Executive Order 14203 authorized asset freezes and visa restrictions against ICC officials investigating U.S. citizens or allies, primarily in response to the ICC’s arrest warrants for Israeli Prime Minister Benjamin Netanyahu over alleged war crimes in Gaza.

Earlier this month the president launched the “Gold Card,” a visa scheme established by an executive order in September that offers wealthy individuals and corporations expedited paths to U.S. residency and citizenship in exchange for $1 million for individuals and $2 million for companies, plus ongoing fees. The administration says it is also planning to offer a “platinum” version of the card that offers special tax breaks — for a cool $5 million.

FEDERAL CYBERSECURITY

President Trump campaigned for a second term insisting that the previous election was riddled with fraud and had been stolen from him. Shortly after Mr. Trump took the oath of office for a second time, he fired the head of the Cybersecurity and Infrastructure Security Agency (CISA) — Chris Krebs (no relation) — for having the audacity to state publicly that the 2020 election was the most secure in U.S. history.

Mr. Trump revoked Krebs’s security clearances, ordered a Justice Department investigation into his election security work, and suspended the security clearances of employees at SentinelOne, the cybersecurity firm where Krebs worked as chief intelligence and public policy officer. The executive order was the first direct presidential action against any US cybersecurity company. Krebs subsequently resigned from SentinelOne, telling The Wall Street Journal he was leaving to push back on Trump’s efforts “to go after corporate interests and corporate relationships.”

The president also dismissed all 15 members of the Cyber Safety Review Board (CSRB), a nonpartisan government entity established in 2022 with a mandate to investigate the security failures behind major cybersecurity events — likely because those advisors included Chris Krebs.

At the time, the CSRB was in the middle of compiling a much-anticipated report on the root causes of Chinese government-backed digital intrusions into at least nine U.S. telecommunications providers. Not to be outdone, the Federal Communication Commission quickly moved to roll back a previous ruling that required U.S. telecom carriers to implement stricter cybersecurity measures.

Meanwhile, CISA has lost roughly a third of its workforce this year amid mass layoffs and deferred resignations. When the government shutdown began in October, CISA laid off even more employees and furloughed 65 percent of the remaining staff, leaving only 900 employees working without pay.

Additionally, the Department of Homeland Security has reassigned CISA cyber specialists to jobs supporting the president’s deportation agenda. As Bloomberg reported earlier this year, CISA employees were given a week to accept the new roles or resign, and some of the reassignments included relocations to new geographic areas.

The White House has signaled that it plans to cut an additional $491 million from CISA’s budget next year, cuts that primarily target CISA programs focused on international affairs and countering misinformation and foreign propaganda. The president’s budget proposal justified the cuts by repeating debunked claims about CISA engaging in censorship.

The Trump administration has pursued a similar reorganization at the FBI: The Washington Post reported in October that a quarter of all FBI agents have now been reassigned from national security threats to immigration enforcement. Reuters reported last week that the replacement of seasoned leaders at the FBI and Justice Department with Trump loyalists has led to an unprecedented number of prosecutorial missteps, resulting in a 21 percent dismissal rate of the D.C. U.S. attorney’s office criminal complaints over eight weeks, compared to a mere .5% dismissal rate over the prior 10 years.

“These mistakes are causing department attorneys to lose credibility with federal courts, with some judges quashing subpoenas, threatening criminal contempt and issuing opinions that raise questions about their conduct,” Reuters reported. “Grand juries have also in some cases started rejecting indictments, a highly unusual event since prosecutors control what evidence gets presented.”

In August, the DHS banned state and local governments from using cyber grants on services provided by the Multi-State Information Sharing and Analysis Center (MS-ISAC), a group that for more than 20 years has shared critical cybersecurity intelligence across state lines and provided software and other resources at free or heavily discounted rates. Specifically, DHS barred states from spending funds on services offered by the Elections Infrastructure ISAC, which was effectively shuttered after DHS pulled its funding in February.

Cybersecurity Dive reports that the Trump administration’s massive workforce cuts, along with widespread mission uncertainty and a persistent leadership void, have interrupted federal agencies’ efforts to collaborate with the businesses and local utilities that run and protect healthcare facilities, water treatment plans, energy companies and telecommunications networks. The publication said the changes came after the US government eliminated CIPAC — a framework that allowed private companies to share cyber and threat intel without legal penalties.

“Government leaders have canceled meetings with infrastructure operators, forced out their longtime points of contact, stopped attending key industry events and scrapped a coordination program that made companies feel comfortable holding sensitive talks about cyberattacks and other threats with federal agencies,” Cybersecurity Dive’s Eric Geller wrote.

Both the National Security Agency (NSA) and U.S. Cyber Command have been without a leader since Trump dismissed Air Force General Timothy Haugh in April, allegedly for disloyalty to the president and at the suggestion of far-right conspiracy theorist Laura Loomer. The nomination of Army Lt. Gen. William Hartman for the same position fell through in October. The White House has ordered the NSA to cut 8 percent of its civilian workforce (between 1,500 and 2,000 employees).

As The Associated Press reported in August, the Office of the Director of National Intelligence plans to dramatically reduce its workforce and cut its budget by more than $700 million annually. Director of National Intelligence Tulsi Gabbard said the cuts were warranted because ODNI had become “bloated and inefficient, and the intelligence community is rife with abuse of power, unauthorized leaks of classified intelligence, and politicized weaponization of intelligence.”

The firing or forced retirements of so many federal employees has been a boon to foreign intelligence agencies. Chinese intelligence agencies, for example, reportedly moved quickly to take advantage of the mass layoffs, using a network of front companies to recruit laid-off U.S. government employees for “consulting work.” Former workers with the Defense Department’s Defense Digital Service who resigned en-masse earlier this year thanks to DOGE encroaching on their mission have been approached by the United Arab Emirates to work on artificial intelligence for the oil kingdom’s armed forces, albeit reportedly with the blessing of the Trump administration.

PRESS FREEDOM

President Trump has filed multibillion-dollar lawsuits against a number of major news outlets over news segments or interviews that allegedly portrayed him in a negative light, suing the networks ABC, the BBC, the CBS parent company Paramount, The Wall Street Journal, and The New York Times, among others.

The president signed an executive order aimed at slashing public subsidies to PBS and NPR, alleging “bias” in the broadcasters’ reporting. In July, Congress approved a request from Trump to cut $1.1 billion in federal funding for the Corporation for Public Broadcasting, the nonprofit entity that funds PBS and NPR.

Brendan Carr, the president’s pick to run the Federal Communications Commission (FCC), initially pledged to “dismantle the censorship cartel and restore free speech rights for everyday Americans.” But on January 22, 2025, the FCC reopened complaints against ABC, CBS and NBC over their coverage of the 2024 election. The previous FCC chair had dismissed the complaints as attacks on the First Amendment and an attempt to weaponize the agency for political purposes.

President Trump in February seized control of the White House Correspondents’ Association, the nonprofit entity that decides which media outlets should have access to the White House and the press pool that follows the president. The president invited an additional 32 media outlets, mostly conservative or right-wing organizations.

According to the journalism group Poynter.org, there are three religious networks, all of which lean conservative, as well as a mix of outlets that includes a legacy paper, television networks, and a digital outlet powered by artificial intelligence.  Trump also barred The Associated Press from the White House over their refusal to refer to the Gulf of Mexico as the Gulf of America.

Under Trump appointee Kari Lake, the U.S. Agency for Global Media moved to dismantle Voice of America, Radio Free Europe/Radio Liberty, and other networks that for decades served as credible news sources behind authoritarian lines. Courts blocked shutdown orders, but the damage continues through administrative leave, contract terminations, and funding disputes.

President Trump this term has fired most of the people involved in processing Freedom of Information Act (FOIA) requests for government agencies. FOIA is an indispensable tool used by journalists and the public to request government records, and to hold leaders accountable.

Petitioning the government, particularly when it ignores your requests, often requires challenging federal agencies in court. But that becomes far more difficult if the most competent law firms start to shy away from cases that may involve crossing the president and his administration. On March 22, the president issued a memorandum that directs heads of the Justice and Homeland Security Departments to “seek sanctions against attorneys and law firms who engage in frivolous, unreasonable and vexatious litigation against the United States,” or in matters that come before federal agencies.

The Trump administration announced increased vetting of applicants for H-1B visas for highly skilled workers, with an internal State Department memo saying that anyone involved in “censorship” of free speech should be considered for rejection.

Executive Order 14161, issued in 2025 on “foreign terrorist and public safety threats,” granted broad new authority that civil rights groups warn could enable a renewed travel ban and expanded visa denials or deportations based on perceived ideology. Critics charged that the order’s vague language around “public safety threats” creates latitude for targeting individuals based on political views, national origin, or religion.

CONSUMER PROTECTION, PRIVACY

At the beginning of this year, President Trump ordered staffers at the Consumer Financial Protection Bureau (CFPB) to stop most work. Created by Congress in 2011 to be a clearinghouse of consumer complaints, the CFPB has sued some of the nation’s largest financial institutions for violating consumer protection laws. The CFPB says its actions have put nearly $18 billion back in Americans’ pockets in the form of monetary compensation or canceled debts, and imposed $4 billion in civil money penalties against violators.

The Trump administration said it planned to fire up to 90 percent of all CFPB staff, but a recent federal appeals court ruling in Washington tossed out an earlier decision that would have allowed the firings to proceed. Reuters reported this week that an employee union and others have battled against it in court for ten months, during which the agency has been almost completely idled.

The CFPB’s acting director is Russell Vought, a key architect of the GOP policy framework Project 2025. Under Vought’s direction, the CFPB in May quietly withdrew a data broker protection rule intended to limit the ability of U.S. data brokers to sell personal information on Americans.

Despite the Federal Reserve’s own post-mortem explicitly blaming Trump-era deregulation for the 2023 Silicon Valley Bank collapse, which triggered a fast-moving crisis requiring emergency weekend bailouts of banks, Trump’s banking regulators in 2025 doubled down. They loosened capital requirements, narrowed definitions of “unsafe” banking practices, and stripped specific risk categories from supervisory frameworks. The setup for another banking crisis requiring taxpayer intervention is now in place.

The Privacy Act of 1974, one of the few meaningful federal privacy laws, was built on the principles of consent and separation in response to the abuses of power that came to light during the Watergate era. The law states that when an individual provides personal information to a federal agency to receive a particular service, that data must be used solely for its original purpose.

Nevertheless, it emerged in June that the Trump administration has built a central database of all US citizens. According to NPR, the White House plans to use the new platform during upcoming elections to verify the identity and citizenship status of US voters. The database was built by the Department of Homeland Security and the Department of Governmental Efficiency and is being rolled out in phases to US states.

DOGE

Probably the biggest ungotten scoop of 2025 is the inside story of what happened to all of the personal, financial and other sensitive data that was accessed by workers at the so-called Department of Government Efficiency (DOGE). President Trump tapped Elon Musk to lead the newly created department, which was mostly populated by current and former employees of Musk’s various technology companies (including a former denizen of the cybercrime community known as the “Com”). It soon emerged that the DOGE team was using artificial intelligence to surveil at least one federal agency’s communications for hostility to Mr. Trump and his agenda.

DOGE employees were able to access and synthesize data taken from a large number of previously separate and highly guarded federal databases, including those at the Social Security Administration, the Department of Homeland Security, the Office of Personnel Management, and the U.S. Department of the Treasury. DOGE staffers did so largely by circumventing or dismantling security measures designed to detect and prevent misuse of federal databases, including standard incident response protocols, auditing, and change-tracking mechanisms.

For example, an IT expert with the National Labor Relations Board (NLRB) alleges that DOGE employees likely downloaded gigabytes of data from agency case files in early March, using short-lived accounts that were configured to leave few traces of network activity. The NLRB whistleblower said the large data outflows coincided with multiple blocked login attempts from addresses in Russia, which attempted to use valid credentials for a newly-created DOGE user account.

The stated goal of DOGE was to reduce bureaucracy and to massively cut costs — mainly by eliminating funding for a raft of federal initiatives that had already been approved by Congress. The DOGE website claimed those efforts reduced “wasteful” and “fraudulent” federal spending by more than $200 billion. However, multiple independent reviews by news organizations determined the true “savings” DOGE achieved was off by a couple of orders of magnitude, and was likely closer to $2 billion.

At the same time DOGE was slashing federal programs, President Trump fired at least 17 inspectors general at federal agencies — the very people tasked with actually identifying and stopping waste, fraud and abuse at the federal level. Those included several agencies (such as the NLRB) that had open investigations into one or more of Mr. Musk’s companies for allegedly failing to comply with protocols aimed at protecting state secrets. In September, a federal judge found the president unlawfully fired the agency watchdogs, but none of them have been reinstated.

Where is DOGE now? Reuters reported last month that as far as the White House is concerned, DOGE no longer exists, even though it technically has more than half a year left to its charter. Meanwhile, who exactly retains access to federal agency data that was fed by DOGE into AI tools is anyone’s guess.

KrebsOnSecurity would like to thank the anonymous researcher NatInfoSec for assisting with the research on this story.

Top

Copying everything off a zpool, destroying it, creating a new one, and copying everything back

Post by Dan Langille via Dan Langille's Other Diary »

I have a problem with a zpool. To be clear, this really isn’t a problem. I’m not aware of any I/O throttling etc. It is just something I would like to change.

zpzpoo%                                                                                                                                                                                                            [18:26 r720-02 dvl ~] % zpool status data01
  pool: data01
 state: ONLINE
status: One or more devices are configured to use a non-native block size.
	Expect reduced performance.
action: Replace affected devices with devices that support the
	configured block size, or migrate data to a properly configured
	pool.
  scan: scrub repaired 0B in 00:15:40 with 0 errors on Mon Dec  8 04:05:38 2025
config:

	NAME                         STATE     READ WRITE CKSUM
	data01                       ONLINE       0     0     0
	  mirror-0                   ONLINE       0     0     0
	    gpt/S59VNS0N809087J_S00  ONLINE       0     0     0  block size: 512B configured, 4096B native
	    gpt/S59VNJ0N631973D_S01  ONLINE       0     0     0  block size: 512B configured, 4096B native
	  mirror-1                   ONLINE       0     0     0
	    gpt/S5B3NDFN807383E_S02  ONLINE       0     0     0  block size: 512B configured, 4096B native
	    gpt/S5B3NDFN807386P_S03  ONLINE       0     0     0
  block size: 512B configured, 4096B native

errors: No known data errors

I point out that this problem was not detected when the zpool was created 5 years ago in Oct 2020. Perhaps this is a situation which ZFS has only recently started to report.

It was discussed on Mastodon and deciding that recreation was the only option.

The plan:

  • copy everything off the zpool
  • destroy the zpool
  • create the zpool
  • copy everything onto the zpool

Sounds easy!

In this post:

  • FreeBSD 14.3

One more time, with a bit of detail

Let me repeat what I just said above, but with slightly more detail.

  • zfs send the data01 zpool from r720-02 (in a data center in New York) to r730-01 (in a rack, in my basement)
  • repeat that send | recv, getting any changes since that initial length copy
  • destroy the zpool on r720-02
  • create the zpool, making sure we have solved the issue
  • copy everything back, using another send | recv

Of note:

  • I will be allowing ssh via root for this purpose. This is not recommended, except in certain temporary circumstances, such as this.
  • ssh will be tightly controlled, by ssh-key only

I’m not going into much detail as to how I’m doing that all.

Allowing root ssh, by key only

I’m setting the bar high here. If you don’t know how to do this, I suggest you don’t want to do this.

I created an ssh key on the sending host. It has a passphrase.

I added that public key to the root account on the receiving host. I enabled root ssh on the receiving host.

All that will be disabled and reversed after I have done this copy/restore. This is not something you should enable forever. It will come back to bite you later.

Creating the destination dataset

This is where we will send data.

root@r730-01:~ # zfs get all data04/r720-02
NAME            PROPERTY              VALUE                  SOURCE
data04/r720-02  type                  filesystem             -
data04/r720-02  creation              Thu Dec 11 18:15 2025  -
data04/r720-02  used                  205K                   -
data04/r720-02  available             17.3T                  -
data04/r720-02  referenced            205K                   -
data04/r720-02  compressratio         1.00x                  -
data04/r720-02  mounted               yes                    -
data04/r720-02  quota                 none                   default
data04/r720-02  reservation           none                   default
data04/r720-02  recordsize            128K                   inherited from data04
data04/r720-02  mountpoint            /data04/r720-02        default
data04/r720-02  sharenfs              off                    default
data04/r720-02  checksum              on                     default
data04/r720-02  compression           zstd                   inherited from data04
data04/r720-02  atime                 on                     default
data04/r720-02  devices               on                     default
data04/r720-02  exec                  on                     default
data04/r720-02  setuid                on                     default
data04/r720-02  readonly              off                    default
data04/r720-02  jailed                off                    default
data04/r720-02  snapdir               hidden                 default
data04/r720-02  aclmode               discard                default
data04/r720-02  aclinherit            restricted             default
data04/r720-02  createtxg             36211                  -
data04/r720-02  canmount              on                     default
data04/r720-02  xattr                 on                     default
data04/r720-02  copies                1                      default
data04/r720-02  version               5                      -
data04/r720-02  utf8only              off                    -
data04/r720-02  normalization         none                   -
data04/r720-02  casesensitivity       sensitive              -
data04/r720-02  vscan                 off                    default
data04/r720-02  nbmand                off                    default
data04/r720-02  sharesmb              off                    default
data04/r720-02  refquota              none                   default
data04/r720-02  refreservation        none                   default
data04/r720-02  guid                  8698413625513271311    -
data04/r720-02  primarycache          all                    default
data04/r720-02  secondarycache        all                    default
data04/r720-02  usedbysnapshots       0B                     -
data04/r720-02  usedbydataset         205K                   -
data04/r720-02  usedbychildren        0B                     -
data04/r720-02  usedbyrefreservation  0B                     -
data04/r720-02  logbias               latency                default
data04/r720-02  objsetid              3399                   -
data04/r720-02  dedup                 off                    default
data04/r720-02  mlslabel              none                   default
data04/r720-02  sync                  standard               default
data04/r720-02  dnodesize             legacy                 default
data04/r720-02  refcompressratio      1.00x                  -
data04/r720-02  written               205K                   -
data04/r720-02  logicalused           42.5K                  -
data04/r720-02  logicalreferenced     42.5K                  -
data04/r720-02  volmode               default                default
data04/r720-02  filesystem_limit      none                   default
data04/r720-02  snapshot_limit        none                   default
data04/r720-02  filesystem_count      none                   default
data04/r720-02  snapshot_count        none                   default
data04/r720-02  snapdev               hidden                 default
data04/r720-02  acltype               nfsv4                  default
data04/r720-02  context               none                   default
data04/r720-02  fscontext             none                   default
data04/r720-02  defcontext            none                   default
data04/r720-02  rootcontext           none                   default
data04/r720-02  relatime              on                     default
data04/r720-02  redundant_metadata    all                    default
data04/r720-02  overlay               on                     default
data04/r720-02  encryption            off                    default
data04/r720-02  keylocation           none                   default
data04/r720-02  keyformat             none                   default
data04/r720-02  pbkdf2iters           0                      default
data04/r720-02  special_small_blocks  0                      default
data04/r720-02  prefetch              all                    default

Starting the send

Here goes:

[root@r720-02:~] # zfs send -vR data01@send-to-r730-01-01 | mbuffer -s 128k -m 1G 2>/dev/null | ssh root@10.55.0.141 'mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02'

Two thoughts:

  • syncoid is not ssh-agent aware from what I can tell
  • I started this in a tmux session

How long?

Now I wait. How long? Let’s estimate.

[15:42 r720-02 dvl ~] % zpool list data01
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  1.81T   759G  1.07T        -         -    30%    40%  1.00x    ONLINE  -

So, 760G to send.

The transfer started at:

[15:42 r720-02 dvl ~] % ps auwwx | grep send
root    77687    1.1  0.0   28004   12324  2  S+   15:36        0:10.75 zfs send -vR data01@send-to-r730-01-01
dvl     81195    0.0  0.0   13836    2400  3  S+   15:43        0:00.00 grep send

15:36 – it is now 15:43 – so 7 minutes.

How much copied?

root@r730-01:~/.ssh # zfs list -r data04/r720-02
NAME                           USED  AVAIL  REFER  MOUNTPOINT
data04/r720-02                2.12G  17.3T   188K  /data04/r720-02
data04/r720-02/jails          2.12G  17.3T   290K  /jails
data04/r720-02/jails/perl540   541M  17.3T   526M  /jails/perl540
data04/r720-02/jails/proxy01   606M  17.3T   606M  /jails/proxy01
data04/r720-02/jails/svn      1.00G  17.3T   984M  /jails/svn

2.12G

Doing the math: 2.12 / 760 = 0.00278 or about 0.3%

0.00278 of X is 7 minutes.

X = 7 / 0.00278 = 2517.98 minutes = 42 hours

That’s how long it should take to get here. Should be done at 4:46 AM on Saturday morning (local time).

Saturday 18:22 UTC

Well, it ain’t done yet.

[18:22 r730-01 dvl ~] % zfs list data04/r720-02 
NAME             USED  AVAIL  REFER  MOUNTPOINT
data04/r720-02   774G  16.5T   188K  /data04/r720-02

774G copied in.

And the copy is still going:

[18:24 r730-01 dvl ~] % ps auwwx | grep zfs
root        7    0.0  0.0       0   37904  -  DL   Tue17      176:19.35 [zfskern]
root    10582    0.0  0.0   14404    2984  -  Is   Fri15        0:00.00 sh -c mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02
root    10584    0.0  0.0   23132   10084  -  S    Fri15       11:13.68 zfs recv -duF data04/r720-02
dvl     20429    0.0  0.0   13836    2392  2  S+   18:24        0:00.00 grep zfs

I’ll check back later.

In the meantime…

I noticed this in recordsize, it’s rather small for the distfiles dataset, which is mostly large tarballs.

[18:24 r730-01 dvl ~] % zfs get -r -t filesystem recordsize data04/r720-02
NAME                                                                   PROPERTY    VALUE    SOURCE
data04/r720-02                                                         recordsize  128K     inherited from data04
data04/r720-02/backups                                                 recordsize  128K     inherited from data04
data04/r720-02/backups/rscyncer                                        recordsize  128K     inherited from data04
data04/r720-02/backups/rscyncer/backups                                recordsize  128K     inherited from data04
data04/r720-02/backups/rscyncer/backups/bacula-database                recordsize  128K     inherited from data04
data04/r720-02/freebsd_releases                                        recordsize  128K     inherited from data04
data04/r720-02/freshports                                              recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01                                    recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/ports                              recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var                                recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db                             recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports                  recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports/cache            recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports/cache/html       recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports/cache/spooling   recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports/message-queues   recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/freshports/repos            recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/ingress                     recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/ingress/message-queues      recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/ingress/repos               recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/ingress_svn                 recordsize  128K     inherited from data04
data04/r720-02/freshports/ingress01/var/db/ingress_svn/message_queues  recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed                                       recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01                             recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01/distfiles                   recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01/jails                       recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01/jails/freshports            recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01/mkjail                      recordsize  128K     inherited from data04
data04/r720-02/freshports/jailed/ingress01/mkjail/14.3-RELEASE         recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01                                      recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var                                  recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db                               recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports                    recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache              recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/categories   recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/commits      recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/daily        recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/general      recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/news         recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/packages     recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/pages        recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/ports        recordsize  128K     inherited from data04
data04/r720-02/freshports/nginx01/var/db/freshports/cache/spooling     recordsize  128K     inherited from data04
data04/r720-02/jails                                                   recordsize  128K     inherited from data04
data04/r720-02/jails/bw                                                recordsize  128K     inherited from data04
data04/r720-02/jails/ingress01                                         recordsize  128K     inherited from data04
data04/r720-02/jails/nginx01                                           recordsize  128K     inherited from data04
data04/r720-02/jails/ns3                                               recordsize  128K     inherited from data04
data04/r720-02/jails/perl540                                           recordsize  128K     inherited from data04
data04/r720-02/jails/pg01                                              recordsize  128K     inherited from data04
data04/r720-02/jails/proxy01                                           recordsize  128K     inherited from data04
data04/r720-02/jails/svn                                               recordsize  128K     inherited from data04
data04/r720-02/mkjail                                                  recordsize  128K     inherited from data04
data04/r720-02/mkjail/14.1-RELEASE                                     recordsize  128K     inherited from data04
data04/r720-02/mkjail/14.2-RELEASE                                     recordsize  128K     inherited from data04
data04/r720-02/mkjail/14.3-RELEASE                                     recordsize  128K     inherited from data04
data04/r720-02/pg01                                                    recordsize  8K       received
data04/r720-02/pg01/dan                                                recordsize  8K       inherited from data04/r720-02/pg01
data04/r720-02/pg01/postgres                                           recordsize  8K       received
data04/r720-02/reserved                                                recordsize  128K     inherited from data04

Let’s fix that on the copy, so when we copy it back, it’s what we want.

I’m going to do some looking, without much to support my decisions. I’m guessing.

We have 17k files:

[18:30 r720-02 dvl /jails/freshports/usr/ports/distfiles] % find . -type f | wc -l
   16885

The biggest out here, seem to be about 2GB:

[18:29 r720-02 dvl /jails/freshports/usr/ports/distfiles] % ls -lSh | head
total 305190300
-rw-r--r--    1 root wheel  2.1G 2024.03.20 12:03 adobe-fonts-source-han-sans-2.001R_GH0.tar.gz
-rw-r--r--    1 root wheel  1.9G 2022.12.23 03:49 adobe-fonts-source-han-sans-2.000R_GH0.tar.gz
-rw-r--r--    1 root wheel  1.8G 2022.12.23 03:54 adobe-fonts-source-han-serif-1.001R_GH0.tar.gz
-rw-r--r--    1 root wheel  1.1G 2023.11.13 04:04 alienarena-alienarena-7.71.6_GH0.tar.gz
-rw-r--r--    1 root wheel  1.1G 2024.08.12 08:03 pycharm-professional-2024.2.tar.gz
-rw-r--r--    1 root wheel  1.1G 2025.02.27 16:50 pycharm-professional-2024.3.4.tar.gz
-rw-r--r--    1 root wheel  1.1G 2025.02.13 16:58 pycharm-professional-2024.3.3.tar.gz
-rw-r--r--    1 root wheel  1.0G 2024.09.26 15:20 pycharm-professional-2024.2.3.tar.gz
-rw-r--r--    1 root wheel  1.0G 2024.09.19 17:33 pycharm-professional-2024.2.2.tar.gz

So I’m going to bump the recordsize to 1M.

[18:26 r730-01 dvl ~] % sudo zfs set recordsize=1M data04/r720-02/freshports/jailed/ingress01/distfiles
[18:34 r730-01 dvl ~] % 

With that, I’m off to the Reindeer Romp.

Sunday morning

The first copy is finished.

...
in @ 8661 kiB/s, out @ 8661 kiB/s,  985 GiB total, buffer   0% full21:04:53    986G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-01
in @ 9547 kiB/s, out @ 9547 kiB/s,  985 GiB total, buffer   0% full21:04:54    986G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-01
TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/Bacula@send-to-r730-01-01
in @ 3564 kiB/s, out @ 3564 kiB/s,  986 GiB total, buffer   0% full
summary:  986 GiByte in 29h 29min 51.4sec - average of 9740 kiB/s
in @ 6125 kiB/s, out @ 6125 kiB/s,  986 GiB total, buffer   0% full
summary:  986 GiByte in 29h 29min 51.3sec - average of 9740 kiB/s

The zpool looks as expected:

% zfs list data04/r720-02
NAME             USED  AVAIL  REFER  MOUNTPOINT
data04/r720-02   803G  16.5T   188K  /data04/r720-02

Initially, I thought it was a little bigger than the source, but:

[root@r720-02:~] # zfs list data01
NAME     USED  AVAIL  REFER  MOUNTPOINT
data01   926G   872G    23K  /data01

No, it’s slightly smaller. As I would hope, with the change from lz4 to zstd for compression.

An incremental send

Here, I do another snapshot:

[root@r720-02:~] # zfs snapshot -r data01@send-to-r730-01-02
[root@r720-02:~] # 

And another send, this time, an incremental. Compared to the previous command (above, I’m adding the -I parameter and specifying a second snapshot name (data01@send-to-r730-01-02, as just taken above). This says: send the diff between these two snapshots…

[root@r720-02:~] # zfs send -vRI data01@send-to-r730-01-01 data01@send-to-r730-01-02 | mbuffer -s 128k -m 1G 2>/dev/null | ssh root@10.55.0.141 'mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02'

Now waiting… it’s going on 2.5 hours…

control-T tells me it’s sent 79.3G so far:

in @ 12.2 MiB/s, out @ 12.2 MiB/s, 78.2 GiB total, buffer   0% full17:40:18   79.2G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 8396 kiB/s, out @ 8396 kiB/s, 78.3 GiB total, buffer   0% full
load: 0.85  cmd: ssh 34552 [running] 8963.27r 553.67u 1437.10s 22% 20936k
17:40:18   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 11.0 MiB/s, out @ 11.0 MiB/s, 78.3 GiB total, buffer   0% full17:40:19   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 9537 kiB/s, out @ 9286 kiB/s, 78.3 GiB total, buffer   0% full17:40:20   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 5561 kiB/s, out @ 5561 kiB/s, 78.3 GiB total, buffer   0% full17:40:21   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 7101 kiB/s, out @ 7101 kiB/s, 78.3 GiB total, buffer   0% full17:40:22   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 8730 kiB/s, out @ 8730 kiB/s, 78.3 GiB total, buffer   0% full17:40:23   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02
in @ 8182 kiB/s, out @ 8182 kiB/s, 78.3 GiB total, buffer 100% full17:40:24   79.3G   data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-02

It’s now 20:24 UTC and we’re done that one:

in @ 10.3 MiB/s, out @ 10.3 MiB/s,  169 GiB total, buffer   0% fullTIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/Bacula@send-to-r730-01-02
in @ 10.8 MiB/s, out @ 10.8 MiB/s,  170 GiB total, buffer   0% full
summary:  170 GiByte in 5h 13min 09.2sec - average of 9488 kiB/s
in @ 9672 kiB/s, out @ 8399 kiB/s,  170 GiB total, buffer   0% full
summary:  170 GiByte in 5h 13min 09.3sec - average of 9487 kiB/s

I’m going to do one more.

[root@r720-02:~] # zfs snapshot -r data01@send-to-r730-01-03
[root@r720-02:~] # zfs send -vRI data01@send-to-r730-01-02 data01@send-to-r730-01-03 | mbuffer -s 128k -m 1G 2>/dev/null | ssh root@10.55.0.141 'mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02'

Let’s see how long this one takes.

Well, not long at all:

TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-03
TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/Bacula@send-to-r730-01-03
in @ 10.6 MiB/s, out @ 10.6 MiB/s, 1080 MiB total, buffer   0% full
summary: 1083 MiByte in  2min 02.9sec - average of 9022 kiB/s
in @  0.0 kiB/s, out @  0.0 kiB/s, 1083 MiB total, buffer   0% full
summary: 1083 MiByte in  2min 05.7sec - average of 8820 kiB/s

Just 2 minutes.

OK, time to shutdown the jails, do a final send, and then destroy/recreate the zpool.

Final copy

Let’s shutdown the jails, and send a last copy over.

[root@r720-02:~] # sudo service jail stop
[root@r720-02:~] # zfs snapshot -r data01@send-to-r730-01-04
[root@r720-02:~] # zfs send -vRI data01@send-to-r730-01-03 data01@send-to-r730-01-04 | mbuffer -s 128k -m 1G 2>/dev/null | ssh root@10.55.0.141 'mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02'

...

TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/bacula-database@send-to-r730-01-04
TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/Bacula@send-to-r730-01-04
in @  0.0 kiB/s, out @ 12.4 MiB/s,  105 MiB total, buffer   0% fullsummary:  105 MiByte in 13.7sec - average of 7867 kiB/s
in @  0.0 kiB/s, out @  755 kiB/s,  105 MiB total, buffer   0% full
summary:  105 MiByte in 16.3sec - average of 6627 kiB/s

Final size check

This is the original:

[root@r720-02:~] # zpool list data01
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  1.81T   831G  1.00T        -         -    31%    44%  1.00x    ONLINE  -

This is the copy:

[14:10 r730-01 dvl ~] % zfs list data04/r720-02 
NAME             USED  AVAIL  REFER  MOUNTPOINT
data04/r720-02   859G  16.4T   188K  /data04/r720-02

And one more:

[root@r720-02:~] # zfs list -r data01 | wc -l
      62
[root@r720-02:~] # 

Compared to:

[20:38 r730-01 dvl ~] % zfs list -r data04/r720-02 | wc -l
      62
[20:38 r730-01 dvl ~] % 

What next?

Next, I’d like to export the zpool, just so I know it’s not being used by anything. Stuff is still mounted and, in my mind, writable. Let’s be sure.

Please watch as I struggle to umount all the stuff.

[root@r720-02:~] # zpool export data01
cannot unmount '/jails/proxy01': pool or dataset is busy

[root@r720-02:~] # mount | grep /jails/proxy01
data01/jails/proxy01 on /jails/proxy01 (zfs, local, nfsv4acls)

[root@r720-02:~] # zfs umount data01/jails/proxy01
cannot unmount '/jails/proxy01': pool or dataset is busy


[root@r720-02:~] # zpool export data01
cannot unmount '/jails/freshports/usr/ports/distfiles': pool or dataset is busy

[root@r720-02:~] # zfs umount -f data01/freshports/jailed/ingress01/distfiles

[root@r720-02:~] # zpool export data01
[root@r720-02:~] # 

There. It’s out. One more snapshot, after importing without mounting:

[root@r720-02:~] # zpool import -N data01
[root@r720-02:~] # zfs snapshot -r data01@send-to-r730-01-05
[root@r720-02:~] # zfs send -vRI data01@send-to-r730-01-04 data01@send-to-r730-01-05 | mbuffer -s 128k -m 1G 2>/dev/null | ssh root@10.55.0.141 'mbuffer -s 128k -m 1G | zfs recv -duF data04/r720-02'
...
TIME        SENT   SNAPSHOT data01/backups/rscyncer/backups/Bacula@send-to-r730-01-05

summary:  702 kiByte in  7.7sec - average of 90.9 kiB/s
in @  0.0 kiB/s, out @  251 kiB/s,  640 kiB total, buffer   0% full
summary:  702 kiByte in 13.2sec - average of 53.1 kiB/s

There, now I’m satisfied. Time to destroy.

[root@r720-02:~] # zpool destroy data01
[root@r720-02:~] # 

Reallocating the drives

I want to reallocate the partitions on those drives.

[root@r720-02:~] # gpart show -l
=>       40  234441568  ada0  GPT  (112G)
         40       1024     1  gptboot0  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  swap0  (2.0G)
    4196352  230244352     3  zfs0  (110G)
  234440704        904        - free -  (452K)

=>       40  234441568  ada1  GPT  (112G)
         40       1024     1  gptboot1  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  swap1  (2.0G)
    4196352  230244352     3  zfs1  (110G)
  234440704        904        - free -  (452K)

=>        40  1953525088  da0  GPT  (932G)
          40  1953520000    1  S59VNS0N809087J_S00  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da1  GPT  (932G)
          40  1953520000    1  S59VNJ0N631973D_S01  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da2  GPT  (932G)
          40  1953520000    1  S5B3NDFN807383E_S02  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da3  GPT  (932G)
          40  1953520000    1  S5B3NDFN807386P_S03  (932G)
  1953520040        5088       - free -  (2.5M)

[root@r720-02:~] # 

It’s da0, da1, da2, and da3.

Let’s see if I can do this without repeating the original mistakes.

Destroy the old partitions:

[root@r720-02:~] # gpart destroy -F da0
da0 destroyed
[root@r720-02:~] # gpart destroy -F da1
da1 destroyed
[root@r720-02:~] # gpart destroy -F da2
da2 destroyed
[root@r720-02:~] # gpart destroy -F da3
da3 destroyed
[root@r720-02:~] # 

One of my favorite scripts. Notice how I use sudo, even when not needed.

[root@r720-02:~] # echo "da0 da1 da2 da3" | xargs -n 1 -I % sudo gpart create -s gpt %
da0 created
da1 created
da2 created
da3 created

And now the partitions.

[root@r720-02:~] # sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 1953520000 -l S59VNS0N809087J_S00 da0
da0p1 added
[root@r720-02:~] # sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 1953520000 -l S59VNJ0N631973D_S01 da1
da1p1 added
[root@r720-02:~] # sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 1953520000 -l S5B3NDFN807383E_S02 da2
da2p1 added
[root@r720-02:~] # gpart add -a 4K -i 1 -t freebsd-zfs -s 1953520000 -l S5B3NDFN807386P_S03 da3
da3p1 added

What do we have?

[root@r720-02:~] # gpart show -l da0 da1 da2 da3
=>        40  1953525088  da0  GPT  (932G)
          40  1953520000    1  S59VNS0N809087J_S00  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da1  GPT  (932G)
          40  1953520000    1  S59VNJ0N631973D_S01  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da2  GPT  (932G)
          40  1953520000    1  S5B3NDFN807383E_S02  (932G)
  1953520040        5088       - free -  (2.5M)

=>        40  1953525088  da3  GPT  (932G)
          40  1953520000    1  S5B3NDFN807386P_S03  (932G)
  1953520040        5088       - free -  (2.5M)
[root@r720-02:~] # 

zpool creation

Let’s try this (taken from Creating a 4 drive ZFS zpool: stripe over mirrors:

[root@r720-02:~] # timtime zpool create data01 \
mirror /dev/gpt/S59VNS0N809087J_S00 /dev/gpt/S59VNJ0N631973D_S01 \
mirror /dev/gpt/S5B3NDFN807383E_S02 /dev/gpt/S5B3NDFN807386P_S03

real    0m0.464s
user    0m0.011s
sys     0m0.048s

[root@r720-02:~] # zpool status data01
  pool: data01
 state: ONLINE
config:

        NAME                         STATE     READ WRITE CKSUM
        data01                       ONLINE       0     0     0
          mirror-0                   ONLINE       0     0     0
            gpt/S59VNS0N809087J_S00  ONLINE       0     0     0
            gpt/S59VNJ0N631973D_S01  ONLINE       0     0     0
          mirror-1                   ONLINE       0     0     0
            gpt/S5B3NDFN807383E_S02  ONLINE       0     0     0
            gpt/S5B3NDFN807386P_S03  ONLINE       0     0     0

errors: No known data errors
[root@r720-02:~] # 

So tell me why this zpool doesn’t have the same problem? What’s the difference between back then and now?

Pulling the data back

This I wasn’t sure about, but I made it up, and it seems to work just fine.

[root@r720-02:~] # ssh root@10.55.0.141 'zfs send -vR data04/r720-02@send-to-r730-01-04 | mbuffer -s 128k -m 1G 2>/dev/null ' | mbuffer -s 128k -m 1G | zfs recv -duF data01

To be fair, I just reconstructed that command from memory and ps auwwwx | grep zfs. It’s already scrolled and gone in my tmux session.

Sounds like I need to come back in 29h 29min.

2025-12-15 – time for traffic graphs

I just updated librenms at home, so let’s look at a graph. This is r720-02 traffic:

r720-02 traffic
r720-02 traffic

The stuff below the line: data leaving the host towards r730-01.

Above the line, data arriving from r730-01.

This graph is not specific to r730-01, but this host, being on standby, does very little other traffic.

Later that same day, I look at this:

[22:21 r720-02 root ~] # date
Mon Dec 15 22:21:53 UTC 2025

[22:21 r720-02 root ~] # zpool list data01
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  1.81T   436G  1.39T        -         -     0%    23%  1.00x    ONLINE  -

So about 50% over.

The tmux session looks like this:

in @ 14.1 MiB/s, out @ 14.1 MiB/s,  811 GiB total, buffer   0% full22:21:40    812G   data04/r720-02/freshports/jailed/ingress01/distfiles@all-distfiles
in @ 11.0 MiB/s, out @ 11.0 MiB/s,  811 GiB total, buffer   0% full22:21:41    812G   data04/r720-02/freshports/jailed/ingress01/distfiles@all-distfiles
in @ 8902 kiB/s, out @ 8902 kiB/s,  811 GiB total, buffer   0% full22:21:42    812G   data04/r720-02/freshports/jailed/ingress01/distfiles@all-distfiles
in @ 9138 kiB/s, out @ 9138 kiB/s,  811 GiB total, buffer   0% full

Time for the book club party. I’ll check back on Tuesday.

It’s done

From what I can tell, the copy is done:

[7:55 pro05 dvl ~] % r720-02
[12:55 r720-02 dvl ~] % zpool list data01
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  1.81T   777G  1.05T        -         -     0%    41%  1.00x    ONLINE  -

[12:55 r720-02 dvl ~] % ps auwwx | grep zfs
root        6    0.0  0.0     0  7280  -  DL   28Nov25     99:20.04 [zfskern]
dvl     13193    0.0  0.0 13836  2096  4  S+   12:55        0:00.00 grep zfs
[12:55 r720-02 dvl ~] % 

Checking in on the tmux session, I found:

...
in @ 8180 kiB/s, out @ 8180 kiB/s, 1154 GiB total, buffer   0% full09:01:22   1.13T   data04/r720-02/freshports/ingress01/ports@send-to-r730-01-03
in @ 11.0 MiB/s, out @ 11.0 MiB/s, 1154 GiB total, buffer   0% full09:01:23   1.13T   data04/r720-02/freshports/ingress01/ports@send-to-r730-01-03
in @ 8912 kiB/s, out @ 8912 kiB/s, 1154 GiB total, buffer   0% full09:01:24   1.13T   data04/r720-02/freshports/ingress01/ports@send-to-r730-01-03
in @ 4810 kiB/s, out @ 4810 kiB/s, 1154 GiB total, buffer   0% fullTIME        SENT   SNAPSHOT data04/r720-02/freshports/ingress01/ports@send-to-r730-01-04
in @ 8180 kiB/s, out @ 8421 kiB/s, 1154 GiB total, buffer   0% full09:01:26   1.13T   data04/r720-02/freshports/ingress01/ports@send-to-r730-01-04
in @  9.9 MiB/s, out @  9.9 MiB/s, 1155 GiB total, buffer   0% full
summary: 1155 GiByte in 35h 41min 22.9sec - average of 9427 kiB/s

There, copied back. Initially I thought the copy back took several hours longer. Then I recalled: the initial send took 29 hours, then a copy more sends were done. I suspect they are close in time.

Some quick validation

[13:05 r730-01 dvl ~] % zfs list -r data04/r720-02 | wc -l
      62
[13:05 r730-01 dvl ~] % zfs list -r -t snapshot data04/r720-02 | wc -l
    1140
[13:06 r720-02 root ~] # zfs list -r data01 | wc -l
      63
[13:07 r720-02 root ~] # zfs list -r -t snapshot data01 | wc -l
    1079

So… that’s unexpected. Mostly, the number the filesystems is concerning. Let’s check.

[13:06 r720-02 root ~] # zfs list -r data01
NAME                                                                    USED  AVAIL  REFER  MOUNTPOINT
data01                                                                  868G   930G    96K  /data01
data01/r720-02                                                          868G   930G    88K  /data01/r720-02
data01/r720-02/backups                                                  145G   930G    96K  none
data01/r720-02/backups/rscyncer                                         145G   930G    96K  none
data01/r720-02/backups/rscyncer/backups                                 145G   930G   112K  /home/rsyncer/backups
data01/r720-02/backups/rscyncer/backups/Bacula                         1.78M   930G  1.50M  /home/rsyncer/backups/Bacula
data01/r720-02/backups/rscyncer/backups/bacula-database                 145G   930G  72.7G  /home/rsyncer/backups/bacula-database
data01/r720-02/freebsd_releases                                         468M   930G   468M  /var/db/mkjail
data01/r720-02/freshports                                               453G   930G    88K  none
data01/r720-02/freshports/ingress01                                    2.89G   930G    96K  none
data01/r720-02/freshports/ingress01/ports                              2.88G   930G  2.53G  /jails/ingress01/jails/freshports/usr/ports
data01/r720-02/freshports/ingress01/var                                1.22M   930G    96K  none
data01/r720-02/freshports/ingress01/var/db                             1.12M   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports                   576K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache             288K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache/html         96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache/spooling     96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/message-queues     96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/repos              96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress                      288K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress/message-queues        96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress/repos                 96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress_svn                  192K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress_svn/message_queues    96K   930G    96K  none
data01/r720-02/freshports/jailed                                        451G   930G    96K  none
data01/r720-02/freshports/jailed/ingress01                              451G   930G    96K  none
data01/r720-02/freshports/jailed/ingress01/distfiles                    449G   930G   449G  /jails/freshports/usr/ports/distfiles
data01/r720-02/freshports/jailed/ingress01/jails                        462M   930G    96K  /jails
data01/r720-02/freshports/jailed/ingress01/jails/freshports             462M   930G   460M  /jails/freshports
data01/r720-02/freshports/jailed/ingress01/mkjail                      1.32G   930G   468M  /var/db/mkjail
data01/r720-02/freshports/jailed/ingress01/mkjail/14.3-RELEASE          880M   930G   868M  /var/db/mkjail/14.3-RELEASE
data01/r720-02/freshports/nginx01                                      2.25M   930G    96K  none
data01/r720-02/freshports/nginx01/var                                  2.16M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db                               2.06M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db/freshports                    1.97M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db/freshports/cache              1.88M   930G    96K  /var/db/freshports/cache
data01/r720-02/freshports/nginx01/var/db/freshports/cache/categories     96K   930G    96K  /var/db/freshports/cache/categories
data01/r720-02/freshports/nginx01/var/db/freshports/cache/commits        96K   930G    96K  /var/db/freshports/cache/commits
data01/r720-02/freshports/nginx01/var/db/freshports/cache/daily          96K   930G    96K  /var/db/freshports/cache/daily
data01/r720-02/freshports/nginx01/var/db/freshports/cache/general        96K   930G    96K  /var/db/freshports/cache/general
data01/r720-02/freshports/nginx01/var/db/freshports/cache/news          288K   930G    96K  /var/db/freshports/cache/news
data01/r720-02/freshports/nginx01/var/db/freshports/cache/packages      264K   930G   128K  /var/db/freshports/cache/packages
data01/r720-02/freshports/nginx01/var/db/freshports/cache/pages          96K   930G    96K  /var/db/freshports/cache/pages
data01/r720-02/freshports/nginx01/var/db/freshports/cache/ports         432K   930G   232K  /var/db/freshports/cache/ports
data01/r720-02/freshports/nginx01/var/db/freshports/cache/spooling      360K   930G   120K  /var/db/freshports/cache/spooling
data01/r720-02/jails                                                    147G   930G   136K  /jails
data01/r720-02/jails/bw                                                3.11G   930G  1.58G  /jails/bw
data01/r720-02/jails/ingress01                                         62.2G   930G  15.9G  /jails/ingress01
data01/r720-02/jails/nginx01                                           8.45G   930G  4.23G  /jails/nginx01
data01/r720-02/jails/ns3                                               7.72G   930G  3.16G  /jails/ns3
data01/r720-02/jails/perl540                                            555M   930G   548M  /jails/perl540
data01/r720-02/jails/pg01                                              59.6G   930G  8.26G  /jails/pg01
data01/r720-02/jails/proxy01                                           3.90G   930G  2.55G  /jails/proxy01
data01/r720-02/jails/svn                                                970M   930G   951M  /jails/svn
data01/r720-02/mkjail                                                  2.56G   930G    96K  /mkjail
data01/r720-02/mkjail/14.1-RELEASE                                      874M   930G   862M  /mkjail/14.1-RELEASE
data01/r720-02/mkjail/14.2-RELEASE                                      868M   930G   856M  /mkjail/14.2-RELEASE
data01/r720-02/mkjail/14.3-RELEASE                                      880M   930G   868M  /mkjail/14.3-RELEASE
data01/r720-02/pg01                                                    28.6G   930G    96K  none
data01/r720-02/pg01/dan                                                  96K   930G    96K  /jails/pg01/usr/home/dan/dump
data01/r720-02/pg01/postgres                                           28.6G   930G  27.8G  /jails/pg01/var/db/postgres
data01/r720-02/reserved                                                91.0G  1021G    96K  /data01/r720-02/reserved

Oh, that’s why. Everything is has one-more-element in the path. I am sure there is on option when sending to avoid this situation.

Each of these locations need to be bumped up by one.

[13:10 r720-02 root ~] # zfs list -d 1 data01/r720-02
NAME                              USED  AVAIL  REFER  MOUNTPOINT
data01/r720-02                    868G   930G    88K  /data01/r720-02
data01/r720-02/backups            145G   930G    96K  none
data01/r720-02/freebsd_releases   468M   930G   468M  /var/db/mkjail
data01/r720-02/freshports         453G   930G    88K  none
data01/r720-02/jails              147G   930G   136K  /jails
data01/r720-02/mkjail            2.56G   930G    96K  /mkjail
data01/r720-02/pg01              28.6G   930G    96K  none
data01/r720-02/reserved          91.0G  1021G    96K  /data01/r720-02/reserved

By that, I mean rename data01/r720-02/backups to data01/backups, and so on.

Renaming

There are only 7 renames to be done, but I want to show how I would code this:

[13:11 r720-02 root ~] # zfs list -o name -H -d 1 data01/r720-02
data01/r720-02
data01/r720-02/backups
data01/r720-02/freebsd_releases
data01/r720-02/freshports
data01/r720-02/jails
data01/r720-02/mkjail
data01/r720-02/pg01
data01/r720-02/reserved

That lists just the datasets.

[13:12 r720-02 root ~] # zfs list -o name -H -d 1 data01/r720-02 | cut -f 3 -d / | grep -v '^$'
backups
freebsd_releases
freshports
jails
mkjail
pg01
reserved

Now we have only the right hand names, and the non-blank values.

Let’s rename:

[13:15 r720-02 root ~] # zfs list -o name -H -d 1 data01/r720-02 | cut -f 3 -d / | grep -v '^$' | xargs -n 1 -I % echo zfs rename data01/r720-02/% data01/%
zfs rename data01/r720-02/backups data01/backups
zfs rename data01/r720-02/freebsd_releases data01/freebsd_releases
zfs rename data01/r720-02/freshports data01/freshports
zfs rename data01/r720-02/jails data01/jails
zfs rename data01/r720-02/mkjail data01/mkjail
zfs rename data01/r720-02/pg01 data01/pg01
zfs rename data01/r720-02/reserved data01/reserved

That looks good enough, let’s really do it now (that was just an echo).

[13:15 r720-02 root ~] # zfs list -o name -H -d 1 data01/r720-02 | cut -f 3 -d / | grep -v '^$' | xargs -n 1 -I % zfs rename data01/r720-02/% data01/%     
cannot rename 'data01/r720-02/freshports': child dataset with inherited mountpoint is used in a non-global zone

That refers to a couple of datasets which are have zfs set jailed set. That jail/unjail should be done in the jail start/stop script. That is for another day.

What we have now is:

[13:17 r720-02 root ~] # zfs list -r data01
NAME                                                                    USED  AVAIL  REFER  MOUNTPOINT
data01                                                                  868G   930G    96K  /data01
data01/backups                                                          145G   930G    96K  none
data01/backups/rscyncer                                                 145G   930G    96K  none
data01/backups/rscyncer/backups                                         145G   930G   112K  /home/rsyncer/backups
data01/backups/rscyncer/backups/Bacula                                 1.78M   930G  1.50M  /home/rsyncer/backups/Bacula
data01/backups/rscyncer/backups/bacula-database                         145G   930G  72.7G  /home/rsyncer/backups/bacula-database
data01/freebsd_releases                                                 468M   930G   468M  /var/db/mkjail
data01/jails                                                            147G   930G   136K  /jails
data01/jails/bw                                                        3.11G   930G  1.58G  /jails/bw
data01/jails/ingress01                                                 62.2G   930G  15.9G  /jails/ingress01
data01/jails/nginx01                                                   8.45G   930G  4.23G  /jails/nginx01
data01/jails/ns3                                                       7.72G   930G  3.16G  /jails/ns3
data01/jails/perl540                                                    555M   930G   548M  /jails/perl540
data01/jails/pg01                                                      59.6G   930G  8.26G  /jails/pg01
data01/jails/proxy01                                                   3.90G   930G  2.55G  /jails/proxy01
data01/jails/svn                                                        970M   930G   951M  /jails/svn
data01/mkjail                                                          2.56G   930G    96K  /mkjail
data01/mkjail/14.1-RELEASE                                              874M   930G   862M  /mkjail/14.1-RELEASE
data01/mkjail/14.2-RELEASE                                              868M   930G   856M  /mkjail/14.2-RELEASE
data01/mkjail/14.3-RELEASE                                              880M   930G   868M  /mkjail/14.3-RELEASE
data01/pg01                                                            28.6G   930G    96K  none
data01/pg01/dan                                                          96K   930G    96K  /jails/pg01/usr/home/dan/dump
data01/pg01/postgres                                                   28.6G   930G  27.8G  /jails/pg01/var/db/postgres
data01/r720-02                                                          453G   930G    88K  /data01/r720-02
data01/r720-02/freshports                                               453G   930G    88K  none
data01/r720-02/freshports/ingress01                                    2.89G   930G    96K  none
data01/r720-02/freshports/ingress01/ports                              2.88G   930G  2.53G  /jails/ingress01/jails/freshports/usr/ports
data01/r720-02/freshports/ingress01/var                                1.22M   930G    96K  none
data01/r720-02/freshports/ingress01/var/db                             1.12M   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports                   576K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache             288K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache/html         96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/cache/spooling     96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/message-queues     96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/freshports/repos              96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress                      288K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress/message-queues        96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress/repos                 96K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress_svn                  192K   930G    96K  none
data01/r720-02/freshports/ingress01/var/db/ingress_svn/message_queues    96K   930G    96K  none
data01/r720-02/freshports/jailed                                        451G   930G    96K  none
data01/r720-02/freshports/jailed/ingress01                              451G   930G    96K  none
data01/r720-02/freshports/jailed/ingress01/distfiles                    449G   930G   449G  /jails/freshports/usr/ports/distfiles
data01/r720-02/freshports/jailed/ingress01/jails                        462M   930G    96K  /jails
data01/r720-02/freshports/jailed/ingress01/jails/freshports             462M   930G   460M  /jails/freshports
data01/r720-02/freshports/jailed/ingress01/mkjail                      1.32G   930G   468M  /var/db/mkjail
data01/r720-02/freshports/jailed/ingress01/mkjail/14.3-RELEASE          880M   930G   868M  /var/db/mkjail/14.3-RELEASE
data01/r720-02/freshports/nginx01                                      2.25M   930G    96K  none
data01/r720-02/freshports/nginx01/var                                  2.16M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db                               2.06M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db/freshports                    1.97M   930G    96K  none
data01/r720-02/freshports/nginx01/var/db/freshports/cache              1.88M   930G    96K  /var/db/freshports/cache
data01/r720-02/freshports/nginx01/var/db/freshports/cache/categories     96K   930G    96K  /var/db/freshports/cache/categories
data01/r720-02/freshports/nginx01/var/db/freshports/cache/commits        96K   930G    96K  /var/db/freshports/cache/commits
data01/r720-02/freshports/nginx01/var/db/freshports/cache/daily          96K   930G    96K  /var/db/freshports/cache/daily
data01/r720-02/freshports/nginx01/var/db/freshports/cache/general        96K   930G    96K  /var/db/freshports/cache/general
data01/r720-02/freshports/nginx01/var/db/freshports/cache/news          288K   930G    96K  /var/db/freshports/cache/news
data01/r720-02/freshports/nginx01/var/db/freshports/cache/packages      264K   930G   128K  /var/db/freshports/cache/packages
data01/r720-02/freshports/nginx01/var/db/freshports/cache/pages          96K   930G    96K  /var/db/freshports/cache/pages
data01/r720-02/freshports/nginx01/var/db/freshports/cache/ports         432K   930G   232K  /var/db/freshports/cache/ports
data01/r720-02/freshports/nginx01/var/db/freshports/cache/spooling      360K   930G   120K  /var/db/freshports/cache/spooling
data01/reserved                                                        91.0G  1021G    96K  /data01/reserved

I know which ones are jailed:

[13:21 r720-02 root ~] # zfs get -t filesystem -r jailed data01/r720-02/freshports | grep ' on '
data01/r720-02/freshports/jailed/ingress01                             jailed    on      received
data01/r720-02/freshports/jailed/ingress01/jails                       jailed    on      received
data01/r720-02/freshports/jailed/ingress01/jails/freshports            jailed    on      inherited from data01/r720-02/freshports/jailed/ingress01/jails
data01/r720-02/freshports/jailed/ingress01/mkjail                      jailed    on      inherited from data01/r720-02/freshports/jailed/ingress01
data01/r720-02/freshports/jailed/ingress01/mkjail/14.3-RELEASE         jailed    on      inherited from data01/r720-02/freshports/jailed/ingress01
data01/r720-02/freshports/nginx01/var/db/freshports                    jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache              jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/categories   jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/commits      jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/daily        jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/general      jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/news         jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/packages     jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/pages        jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/ports        jailed    on      received
data01/r720-02/freshports/nginx01/var/db/freshports/cache/spooling     jailed    on      received

This, I did manually:

[13:24 r720-02 root ~] # history 0 | grep 'zfs set jailed'
   29  zfs set jailed=off data01/r720-02/freshports/jailed/ingress01
   31  zfs set jailed=off data01/r720-02/freshports/jailed/ingress01/jails
   33  zfs set jailed=off data01/r720-02/freshports/jailed/nginx01/var/db/freshports
   34  zfs set jailed=off data01/r720-02/freshports//nginx01/var/db/freshports
   35  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports
   37  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache
   39  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/categories
   40  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/commits
   41  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/daily
   42  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/general
   43  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/news
   44  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/packages
   45  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/pages
   46  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/ports
   47  zfs set jailed=off data01/r720-02/freshports/nginx01/var/db/freshports/cache/spooling

One more rename:

[13:25 r720-02 root ~] # zfs rename data01/r720-02/freshports data01/freshports 

And a destroy:

[13:26 r720-02 root ~] # zfs destroy data01/r720-02
cannot destroy 'data01/r720-02': filesystem has children
use '-r' to destroy the following datasets:
data01/r720-02@send-to-r730-01-01
data01/r720-02@send-to-r730-01-02
data01/r720-02@send-to-r730-01-03
data01/r720-02@send-to-r730-01-04
[13:26 r720-02 root ~] # zfs destroy -r data01/r720-02
[13:26 r720-02 root ~] # 

Now we have:

[13:26 r720-02 root ~] # zfs list -r data01           
NAME                                                            USED  AVAIL  REFER  MOUNTPOINT
data01                                                          868G   930G    96K  /data01
data01/backups                                                  145G   930G    96K  none
data01/backups/rscyncer                                         145G   930G    96K  none
data01/backups/rscyncer/backups                                 145G   930G   112K  /home/rsyncer/backups
data01/backups/rscyncer/backups/Bacula                         1.78M   930G  1.50M  /home/rsyncer/backups/Bacula
data01/backups/rscyncer/backups/bacula-database                 145G   930G  72.7G  /home/rsyncer/backups/bacula-database
data01/freebsd_releases                                         468M   930G   468M  /var/db/mkjail
data01/freshports                                               453G   930G    88K  none
data01/freshports/ingress01                                    2.89G   930G    96K  none
data01/freshports/ingress01/ports                              2.88G   930G  2.53G  /jails/ingress01/jails/freshports/usr/ports
data01/freshports/ingress01/var                                1.22M   930G    96K  none
data01/freshports/ingress01/var/db                             1.12M   930G    96K  none
data01/freshports/ingress01/var/db/freshports                   576K   930G    96K  none
data01/freshports/ingress01/var/db/freshports/cache             288K   930G    96K  none
data01/freshports/ingress01/var/db/freshports/cache/html         96K   930G    96K  none
data01/freshports/ingress01/var/db/freshports/cache/spooling     96K   930G    96K  none
data01/freshports/ingress01/var/db/freshports/message-queues     96K   930G    96K  none
data01/freshports/ingress01/var/db/freshports/repos              96K   930G    96K  none
data01/freshports/ingress01/var/db/ingress                      288K   930G    96K  none
data01/freshports/ingress01/var/db/ingress/message-queues        96K   930G    96K  none
data01/freshports/ingress01/var/db/ingress/repos                 96K   930G    96K  none
data01/freshports/ingress01/var/db/ingress_svn                  192K   930G    96K  none
data01/freshports/ingress01/var/db/ingress_svn/message_queues    96K   930G    96K  none
data01/freshports/jailed                                        451G   930G    96K  none
data01/freshports/jailed/ingress01                              451G   930G    96K  none
data01/freshports/jailed/ingress01/distfiles                    449G   930G   449G  /jails/freshports/usr/ports/distfiles
data01/freshports/jailed/ingress01/jails                        462M   930G    96K  /jails
data01/freshports/jailed/ingress01/jails/freshports             462M   930G   460M  /jails/freshports
data01/freshports/jailed/ingress01/mkjail                      1.32G   930G   468M  /var/db/mkjail
data01/freshports/jailed/ingress01/mkjail/14.3-RELEASE          880M   930G   868M  /var/db/mkjail/14.3-RELEASE
data01/freshports/nginx01                                      2.25M   930G    96K  none
data01/freshports/nginx01/var                                  2.16M   930G    96K  none
data01/freshports/nginx01/var/db                               2.06M   930G    96K  none
data01/freshports/nginx01/var/db/freshports                    1.97M   930G    96K  none
data01/freshports/nginx01/var/db/freshports/cache              1.88M   930G    96K  /var/db/freshports/cache
data01/freshports/nginx01/var/db/freshports/cache/categories     96K   930G    96K  /var/db/freshports/cache/categories
data01/freshports/nginx01/var/db/freshports/cache/commits        96K   930G    96K  /var/db/freshports/cache/commits
data01/freshports/nginx01/var/db/freshports/cache/daily          96K   930G    96K  /var/db/freshports/cache/daily
data01/freshports/nginx01/var/db/freshports/cache/general        96K   930G    96K  /var/db/freshports/cache/general
data01/freshports/nginx01/var/db/freshports/cache/news          288K   930G    96K  /var/db/freshports/cache/news
data01/freshports/nginx01/var/db/freshports/cache/packages      264K   930G   128K  /var/db/freshports/cache/packages
data01/freshports/nginx01/var/db/freshports/cache/pages          96K   930G    96K  /var/db/freshports/cache/pages
data01/freshports/nginx01/var/db/freshports/cache/ports         432K   930G   232K  /var/db/freshports/cache/ports
data01/freshports/nginx01/var/db/freshports/cache/spooling      360K   930G   120K  /var/db/freshports/cache/spooling
data01/jails                                                    147G   930G   136K  /jails
data01/jails/bw                                                3.11G   930G  1.58G  /jails/bw
data01/jails/ingress01                                         62.2G   930G  15.9G  /jails/ingress01
data01/jails/nginx01                                           8.45G   930G  4.23G  /jails/nginx01
data01/jails/ns3                                               7.72G   930G  3.16G  /jails/ns3
data01/jails/perl540                                            555M   930G   548M  /jails/perl540
data01/jails/pg01                                              59.6G   930G  8.26G  /jails/pg01
data01/jails/proxy01                                           3.90G   930G  2.55G  /jails/proxy01
data01/jails/svn                                                970M   930G   951M  /jails/svn
data01/mkjail                                                  2.56G   930G    96K  /mkjail
data01/mkjail/14.1-RELEASE                                      874M   930G   862M  /mkjail/14.1-RELEASE
data01/mkjail/14.2-RELEASE                                      868M   930G   856M  /mkjail/14.2-RELEASE
data01/mkjail/14.3-RELEASE                                      880M   930G   868M  /mkjail/14.3-RELEASE
data01/pg01                                                    28.6G   930G    96K  none
data01/pg01/dan                                                  96K   930G    96K  /jails/pg01/usr/home/dan/dump
data01/pg01/postgres                                           28.6G   930G  27.8G  /jails/pg01/var/db/postgres
data01/reserved                                                91.0G  1021G    96K  /data01/reserved

[13:27 r720-02 root ~] # zfs list -r data01 | wc -l
      62

Now we have the expected number of filesystems.

[13:28 r720-02 root ~] # zfs list -t snapshot -r data01 | wc -l
    1079

One tweak

I want to rename one more dataset:

[13:30 r720-02 root ~] # zfs rename data01/freshports/nginx01 data01/freshports/jailed/nginx01
[13:30 r720-02 root ~] # 

Now all the jailed datasets related to FreshPorts are under one common parent: data01/freshports/jailed. This is a self-imposed naming convention established after this host was deployed.

I have this nagging feeling that it wasn’t there already for a good reason. Which isn’t coming to mind right now. This seems like a good situation for user-supplied notes on that dataset. If only I’d done that then…

Am I ready to start the jails?

Yes, I think the host is ready, but work starts soon, so I’ll delay the jail start until I have more time to follow up on any issues which might arise.

Now it’s time…

Let’s try this:

[root@r720-02:~] # service jail start nginx01
Cannot 'start' jail. Set jail_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.
[root@r720-02:~] # service jail onestart nginx01
Starting jails: cannot start jail  "nginx01": 
jail: pg01: mount.devfs: /jails/pg01/dev: No such file or directory
.

The nginx01 jail needs the database server running. It can’t start because that jail is not mounted.

[root@r720-02:~] # zfs get -r -t filesystem mounted data01
NAME                                                                 PROPERTY  VALUE    SOURCE
data01                                                               mounted   yes      -
data01/backups                                                       mounted   no       -
data01/backups/rscyncer                                              mounted   no       -
data01/backups/rscyncer/backups                                      mounted   no       -
data01/backups/rscyncer/backups/Bacula                               mounted   no       -
data01/backups/rscyncer/backups/bacula-database                      mounted   no       -
data01/freebsd_releases                                              mounted   no       -
data01/freshports                                                    mounted   no       -
data01/freshports/ingress01                                          mounted   no       -
data01/freshports/ingress01/ports                                    mounted   no       -
data01/freshports/ingress01/var                                      mounted   no       -
data01/freshports/ingress01/var/db                                   mounted   no       -
data01/freshports/ingress01/var/db/freshports                        mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache                  mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache/html             mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache/spooling         mounted   no       -
data01/freshports/ingress01/var/db/freshports/message-queues         mounted   no       -
data01/freshports/ingress01/var/db/freshports/repos                  mounted   no       -
data01/freshports/ingress01/var/db/ingress                           mounted   no       -
data01/freshports/ingress01/var/db/ingress/message-queues            mounted   no       -
data01/freshports/ingress01/var/db/ingress/repos                     mounted   no       -
data01/freshports/ingress01/var/db/ingress_svn                       mounted   no       -
data01/freshports/ingress01/var/db/ingress_svn/message_queues        mounted   no       -
data01/freshports/jailed                                             mounted   no       -
data01/freshports/jailed/ingress01                                   mounted   no       -
data01/freshports/jailed/ingress01/distfiles                         mounted   no       -
data01/freshports/jailed/ingress01/jails                             mounted   no       -
data01/freshports/jailed/ingress01/jails/freshports                  mounted   no       -
data01/freshports/jailed/ingress01/mkjail                            mounted   no       -
data01/freshports/jailed/ingress01/mkjail/14.3-RELEASE               mounted   no       -
data01/freshports/jailed/nginx01                                     mounted   no       -
data01/freshports/jailed/nginx01/var                                 mounted   no       -
data01/freshports/jailed/nginx01/var/db                              mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports                   mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache             mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/categories  mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/commits     mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/daily       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/general     mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/news        mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/packages    mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/pages       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/ports       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/spooling    mounted   no       -
data01/jails                                                         mounted   no       -
data01/jails/bw                                                      mounted   no       -
data01/jails/ingress01                                               mounted   no       -
data01/jails/nginx01                                                 mounted   no       -
data01/jails/ns3                                                     mounted   no       -
data01/jails/perl540                                                 mounted   no       -
data01/jails/pg01                                                    mounted   no       -
data01/jails/proxy01                                                 mounted   no       -
data01/jails/svn                                                     mounted   no       -
data01/mkjail                                                        mounted   no       -
data01/mkjail/14.1-RELEASE                                           mounted   no       -
data01/mkjail/14.2-RELEASE                                           mounted   no       -
data01/mkjail/14.3-RELEASE                                           mounted   no       -
data01/pg01                                                          mounted   no       -
data01/pg01/dan                                                      mounted   no       -
data01/pg01/postgres                                                 mounted   no       -
data01/reserved                                                      mounted   no       -

Nothing of use is mounted. I think I know why. I did a zpool import -N data01 before receiving the zpool back.

Now I’ll do this:

[root@r720-02:~] # zpool export data01
[root@r720-02:~] # zpool import data01
[root@r720-02:~] # zfs get -r -t filesystem mounted data01
NAME                                                                 PROPERTY  VALUE    SOURCE
data01                                                               mounted   yes      -
data01/backups                                                       mounted   no       -
data01/backups/rscyncer                                              mounted   no       -
data01/backups/rscyncer/backups                                      mounted   yes      -
data01/backups/rscyncer/backups/Bacula                               mounted   yes      -
data01/backups/rscyncer/backups/bacula-database                      mounted   yes      -
data01/freebsd_releases                                              mounted   yes      -
data01/freshports                                                    mounted   no       -
data01/freshports/ingress01                                          mounted   no       -
data01/freshports/ingress01/ports                                    mounted   no       -
data01/freshports/ingress01/var                                      mounted   no       -
data01/freshports/ingress01/var/db                                   mounted   no       -
data01/freshports/ingress01/var/db/freshports                        mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache                  mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache/html             mounted   no       -
data01/freshports/ingress01/var/db/freshports/cache/spooling         mounted   no       -
data01/freshports/ingress01/var/db/freshports/message-queues         mounted   no       -
data01/freshports/ingress01/var/db/freshports/repos                  mounted   no       -
data01/freshports/ingress01/var/db/ingress                           mounted   no       -
data01/freshports/ingress01/var/db/ingress/message-queues            mounted   no       -
data01/freshports/ingress01/var/db/ingress/repos                     mounted   no       -
data01/freshports/ingress01/var/db/ingress_svn                       mounted   no       -
data01/freshports/ingress01/var/db/ingress_svn/message_queues        mounted   no       -
data01/freshports/jailed                                             mounted   no       -
data01/freshports/jailed/ingress01                                   mounted   no       -
data01/freshports/jailed/ingress01/distfiles                         mounted   no       -
data01/freshports/jailed/ingress01/jails                             mounted   no       -
data01/freshports/jailed/ingress01/jails/freshports                  mounted   no       -
data01/freshports/jailed/ingress01/mkjail                            mounted   no       -
data01/freshports/jailed/ingress01/mkjail/14.3-RELEASE               mounted   no       -
data01/freshports/jailed/nginx01                                     mounted   no       -
data01/freshports/jailed/nginx01/var                                 mounted   no       -
data01/freshports/jailed/nginx01/var/db                              mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports                   mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache             mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/categories  mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/commits     mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/daily       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/general     mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/news        mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/packages    mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/pages       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/ports       mounted   no       -
data01/freshports/jailed/nginx01/var/db/freshports/cache/spooling    mounted   no       -
data01/jails                                                         mounted   yes      -
data01/jails/bw                                                      mounted   yes      -
data01/jails/ingress01                                               mounted   yes      -
data01/jails/nginx01                                                 mounted   yes      -
data01/jails/ns3                                                     mounted   yes      -
data01/jails/perl540                                                 mounted   yes      -
data01/jails/pg01                                                    mounted   yes      -
data01/jails/proxy01                                                 mounted   yes      -
data01/jails/svn                                                     mounted   yes      -
data01/mkjail                                                        mounted   yes      -
data01/mkjail/14.1-RELEASE                                           mounted   yes      -
data01/mkjail/14.2-RELEASE                                           mounted   yes      -
data01/mkjail/14.3-RELEASE                                           mounted   yes      -
data01/pg01                                                          mounted   no       -
data01/pg01/dan                                                      mounted   yes      -
data01/pg01/postgres                                                 mounted   yes      -
data01/reserved                                                      mounted   yes      -

Now we have mounted stuff.

Let’s try again:

[root@r720-02:~] # service jail onestartart nginx01
Starting jails: nginx01.
[root@r720-02:~] # jls
   JID  IP Address      Hostname                      Path
     9  127.163.54.32   r720-02-pg01.int.unixathome.o /jails/pg01
    10  127.163.0.80    r720-02-nginx01.int.unixathom /jails/nginx01
[root@r720-02:~] # 

That looks better. Let’s go:

[root@r720-02:~] # service jail onestartart nginx01
Starting jails: nginx01.
[root@r720-02:~] # jls
   JID  IP Address      Hostname                      Path
     9  127.163.54.32   r720-02-pg01.int.unixathome.o /jails/pg01
    10  127.163.0.80    r720-02-nginx01.int.unixathom /jails/nginx01
[root@r720-02:~] # 

Good. And the webpage loads up, despite it being 2 days out of date. Let’s try the other jails.

[root@r720-02:~] # service jail onestart bw ns3 proxy01
Starting jails: bw ns3 proxy01.
[root@r720-02:~] # 

Now I’ll wait a bit, before starting up the ingress jail. Let the monitoring figure things out…

Then, I’ll disable services within ingress01 before starting it up. That may allow me to work out any filesystem issues before commit processing resumes on that host.`

It’s done

This has been a long rambling post. There were many things rather specific to my configuration and many tweaks to be carried out after the zpool was copied back.

As of now, everything is running as expected.

Notes from ZFS Production Users Call

[22:39 r720-02 dvl ~] % sysctl -a | grep ashift
vfs.zfs.max_auto_ashift: 14
vfs.zfs.min_auto_ashift: 9
vfs.zfs.vdev.max_auto_ashift: 14
vfs.zfs.vdev.min_auto_ashift: 9
vfs.zfs.vdev.file.physical_ashift: 9
vfs.zfs.vdev.file.logical_ashift: 9

This is just notes for me – I’ll clean this up later.

[22:44 r730-01 dvl ~] % sudo zdb -C data01 | grep ashift
ashift: 12

Interesting stuff:

[22:40 r720-02 dvl ~] % sudo zdb -C data01 | less

MOS Configuration:
version: 5000
name: ‘data01’
state: 0
txg: 34131
pool_guid: 582701374551249018
errata: 0
hostid: 1459222754
….

Top

FreeBSD Closes the Laptop Gap: Year One Project Update

Post by FreeBSD Foundation via FreeBSD Foundation »

If you’ve been waiting for the right moment to try FreeBSD on a laptop, take note – 2025 has brought transformative changes. The Foundation’s ambitious Laptop Support & Usability Project is systematically addressing the gaps that have held FreeBSD back on modern laptop hardware

The project started in 2024 Q4 and covers areas including Wi-Fi, graphics, audio, installer, and sleep states. 2025 has been its first full year, and with a financial commitment of over $750k to date there has been substantial progress.

Wi-Fi

In 2025 we added support for W-iFi 4 and 5 on key hardware and made a start on Wi-Fi 6. Wi-Fi 4 and 5 drivers for Intel and Realtek are available in 15.0, with support for additional Realtek and Mediatek drivers in progress. Thanks to Bjoern Zeeb and Tom Jones for their work on this.

Graphics

Graphics drivers have been upgraded to Linux 6.9, which is available in 15.0 (note: to use this driver you need to adopt the drm-latest-kmod port as it’s a non-LTS version). Linux 6.10 is in progress, and this is the minimum version required for the latest Framework laptop (16″ AMD Ryzen AI 300 series). Thanks to Jean-Sébastien Pédron for all the hard work.

Audio

Notable updates to the audio experience are also available in 15.0. Two new utilities; sndctl(8) and mididump(1), several bug fixes and improvements, wider laptop support, as well as an initial attempt to address automatic sound redirection for HDA sound cards. Our thanks go out to Christos Margiolis for his work on this.

Installer

The installer for FreeBSD has gained a couple of new features that benefit laptop users. In 15.0 the installer now supports downloading and installing firmware packages after the FreeBSD base system installation is complete. Coming in 15.1 it will be possible to install the KDE graphical desktop environment during the installation process. Grateful thanks to Bjoern Zeeb and Alfonso Siciliano respectively.

Sleep states

Important work is underway on enabling modern standby (S0i3) which should be available in 15.1, and hibernate (S4) which may be available in 15.2. Additional related areas of functionality (for example, managing the transition from modern standby to hibernate, disk encryption on hibernate, and sleep states for VMs) will be addressed as part of the S4 work. Thanks to Aymeric Wibo, Tom Jones, En-Wei Wu, Konstantin Belousov, and Olivier Certner for their work on sleep states.

2026 plans

The project continues into 2026 with a similar sized investment and scope. Key targets include completing work on sleep states (modern standby and hibernate), adding support for graphics drivers up to Linux 6.18, Wi-Fi 6 support, USB4 and Thunderbolt support, HDMI improvements, UVC webcam support, and Bluetooth improvements.

 

A substantial testing program will also start in January, aiming to test all the functionality together across a range of hardware. Community testers are very welcome to help out, the Foundation will release a blog post and send an invite to help to the Desktop mailing list some time in January 2026.

A Community Effort

This year reinforced to us that meaningful progress only happens when communities, contributors, and organizations share responsibility for sustaining the work that supports us all.

Thank you to all the developers, volunteers, testers, donors, and advocates who contributed to this project. Your support is helping build a more modern, usable, and inclusive FreeBSD ecosystem.

Learn more about the Foundation’s Laptop Project at https://github.com/FreeBSDFoundation/proj-laptop 

Support This Work

Continued progress on laptop support, usability, and hardware enablement is made possible through community donations.

You can support ongoing improvements here:
Donate to FreeBSD Foundation | FreeBSD Foundation

– Contributed by Alice Sowerby

The post FreeBSD Closes the Laptop Gap: Year One Project Update first appeared on FreeBSD Foundation.

Top

Most Parked Domains Now Serving Malicious Content

Post by Brian Krebs via Krebs on Security »

Direct navigation — the act of visiting a website by manually typing a domain name in a web browser — has never been riskier: A new study finds the vast majority of “parked” domains — mostly expired or dormant domain names, or common misspellings of popular websites — are now configured to redirect visitors to sites that foist scams and malware.

A lookalike domain to the FBI Internet Crime Complaint Center website, returned a non-threatening parking page (left) whereas a mobile user was instantly directed to deceptive content in October 2025 (right). Image: Infoblox.

When Internet users try to visit expired domain names or accidentally navigate to a lookalike “typosquatting” domain, they are typically brought to a placeholder page at a domain parking company that tries to monetize the wayward traffic by displaying links to a number of third-party websites that have paid to have their links shown.

A decade ago, ending up at one of these parked domains came with a relatively small chance of being redirected to a malicious destination: In 2014, researchers found (PDF) that parked domains redirected users to malicious sites less than five percent of the time — regardless of whether the visitor clicked on any links at the parked page.

But in a series of experiments over the past few months, researchers at the security firm Infoblox say they discovered the situation is now reversed, and that malicious content is by far the norm now for parked websites.

“In large scale experiments, we found that over 90% of the time, visitors to a parked domain would be directed to illegal content, scams, scareware and anti-virus software subscriptions, or malware, as the ‘click’ was sold from the parking company to advertisers, who often resold that traffic to yet another party,” Infoblox researchers wrote in a paper published today.

Infoblox found parked websites are benign if the visitor arrives at the site using a virtual private network (VPN), or else via a non-residential Internet address. For example, Scotiabank.com customers who accidentally mistype the domain as scotaibank[.]com will see a normal parking page if they’re using a VPN, but will be redirected to a site that tries to foist scams, malware or other unwanted content if coming from a residential IP address. Again, this redirect happens just by visiting the misspelled domain with a mobile device or desktop computer that is using a residential IP address.

According to Infoblox, the person or entity that owns scotaibank[.]com has a portfolio of nearly 3,000 lookalike domains, including gmai[.]com, which demonstrably has been configured with its own mail server for accepting incoming email messages. Meaning, if you send an email to a Gmail user and accidentally omit the “l” from “gmail.com,” that missive doesn’t just disappear into the ether or produce a bounce reply: It goes straight to these scammers. The report notices this domain also has been leveraged in multiple recent business email compromise campaigns, using a lure indicating a failed payment with trojan malware attached.

Infoblox found this particular domain holder (betrayed by a common DNS server — torresdns[.]com) has set up typosquatting domains targeting dozens of top Internet destinations, including Craigslist, YouTube, Google, Wikipedia, Netflix, TripAdvisor, Yahoo, eBay, and Microsoft. A defanged list of these typosquatting domains is available here (the dots in the listed domains have been replaced with commas).

David Brunsdon, a threat researcher at Infoblox, said the parked pages send visitors through a chain of redirects, all while profiling the visitor’s system using IP geolocation, device fingerprinting, and cookies to determine where to redirect domain visitors.

“It was often a chain of redirects — one or two domains outside the parking company — before threat arrives,” Brunsdon said. “Each time in the handoff the device is profiled again and again, before being passed off to a malicious domain or else a decoy page like Amazon.com or Alibaba.com if they decide it’s not worth targeting.”

Brunsdon said domain parking services claim the search results they return on parked pages are designed to be relevant to their parked domains, but that almost none of this displayed content was related to the lookalike domain names they tested.

Samples of redirection paths when visiting scotaibank dot com. Each branch includes a series of domains observed, including the color-coded landing page. Image: Infoblox.

Infoblox said a different threat actor who owns domaincntrol[.]com — a domain that differs from GoDaddy’s name servers by a single character — has long taken advantage of typos in DNS configurations to drive users to malicious websites. In recent months, however, Infoblox discovered the malicious redirect only happens when the query for the misconfigured domain comes from a visitor who is using Cloudflare’s DNS resolvers (1.1.1.1), and that all other visitors will get a page that refuses to load.

The researchers found that even variations on well-known government domains are being targeted by malicious ad networks.

“When one of our researchers tried to report a crime to the FBI’s Internet Crime Complaint Center (IC3), they accidentally visited ic3[.]org instead of ic3[.]gov,” the report notes. “Their phone was quickly redirected to a false ‘Drive Subscription Expired’ page. They were lucky to receive a scam; based on what we’ve learnt, they could just as easily receive an information stealer or trojan malware.”

The Infoblox report emphasizes that the malicious activity they tracked is not attributed to any known party, noting that the domain parking or advertising platforms named in the study were not implicated in the malvertising they documented.

However, the report concludes that while the parking companies claim to only work with top advertisers, the traffic to these domains was frequently sold to affiliate networks, who often resold the traffic to the point where the final advertiser had no business relationship with the parking companies.

Infoblox also pointed out that recent policy changes by Google may have inadvertently increased the risk to users from direct search abuse. Brunsdon said Google Adsense previously defaulted to allowing their ads to be placed on parked pages, but that in early 2025 Google implemented a default setting that had their customers opt-out by default on presenting ads on parked domains — requiring the person running the ad to voluntarily go into their settings and turn on parking as a location.

Top

Installing a 32k DS1386 into an SGI Indy that expects an 8k DS1386

Post by Adrian Chadd via Adrian Chadd's Ramblings »

 The TL;DR is - you can't just plug a 32k DS1386 into an SGI Indy and have it work, as a bunch of stuff is wrong.

 The longer version!

Here's a picture from the datasheet:

Now, at the outset it looks like A13 and A14 on the 32k module need to at least be grounded - if they're floated or high then the RAM will be selected when you don't want it to be, and you'll just fail to see the RTC.

 However, if you just do that your Indy won't boot because it turns out that pin 3 on the 8k module is an undocumented (in this datasheet) auxiliary Vcc input - the 5v auxiliary / always-on rail is actually there. Shorting it to ground will just make the Indy super sad. Don't do that.

So, in lieu of making up a PCB (which I think I'm going to do anyway just to be "clean" - I used some pin headers to raise the DS1386 above the Indy PCB.


 Note that pin 3 on and pin 28 don't have pins (and I'm going to put some tape over the pins just to be super clear nothing shorts out).

Then, I did a quick bodge wire job on the underside of the DS1386-32K module:


 Where pin 3 and pin 28 are tied to pin 16.

 Then, well, plug it in, align it right, and it should just work! It did for me!

Top

Post by FreeBSD Newsflash via FreeBSD News Flash »

New committer: Johan Söllvander (src)
Top

Valuable News – 2025/12/15

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

One Too Many Words on AT&T $2000 Korn Shell and Other Usenet Topics.
https://blog.gabornyeki.com/2025-12-usenet/

Bastillion is Modern Web Based SSH Console and Key Management Tool.
https://github.com/bastillion-io/Bastillion

How to Configure and Enable SSH on FreeBSD 15.
https://ostechnix.com/how-to-enable-ssh-on-freebsd/

Ventoy 1.1.08 Adds Support for FreeBSD 15.0.
https://linuxiac.com/ventoy-1-1-08-adds-support-for-freebsd-15-0/

How to Support FreeBSD on Your Cloud.
https://daemonology.net/blog/2025-11-08-How-to-support-FreeBSD-on-your-cloud.html

University of Utah Discovers Original UNIX v4 OS from 1973.
https://ksltv.com/science-technology/university-of-utah-discovers-rare-computer-relic/853296/

FreeBSD Git Weekly: 2025-12-01 to 2025-12-07.
https://freebsd-git-weekly.tarsnap.net/2025-12-01.html

FreeBSD 15.0 Post Upgrade Reboot Loop.
https://neosmart.net/blog/freebsd-15-0-post-upgrade-reboot-loop/

Benchmarking Rust Compilation Speedups/Slowdowns from sccache(1) and -Zthreads Option.
https://neosmart.net/blog/benchmarking-rust-compilation-speedups-and-slowdowns-from-sccache-and-zthreads/

Misconfigured Docker Instances Used as Botnet to Mine Crypto.
https://youtube.com/watch?v=aT4DBtD7lYc

Building Self Hosted CDN for BSD Cafe Media.
https://it-notes.dragas.net/2024/08/26/building-a-self-hosted-cdn-for-bsd-cafe-media/

Add/Delete/Grant sudo(8) Privileges to Users in FreeBSD.
https://ostechnix.com/add-delete-and-grant-sudo-privileges-to-users-in-freebsd/

FreeBSD/Apache/MySQL/PHP in FreeBSD Jail.
https://youtube.com/watch?v=g5s-s6LdnPE

FreeBSD 15.0 vs. Ubuntu for AMD EPYC Server Performance.
https://phoronix.com/review/freebsd-15-amd-epyc-linux

How to Test SSD Read and Write Speed in Linux.
https://pbxscience.com/how-to-test-ssd-read-and-write-speed-in-linux/

OpenBSD Stories – Story of Propolice.
http://miod.online.fr/software/openbsd/stories/propolice.html

WINE Gaming in FreeBSD Jails with BastilleBSD.
https://pertho.net/2025/11/07/wine-gaming-freebsd-jails/

Reviving 10 Years Old Laptop with FreeBSD.
https://pertho.net/2022/04/19/old-laptop-gaming-freebsd/

Running FreeBSD VNET Jails on AWS EC2 with BastilleBSD.
https://pertho.net/2022/06/06/bastille-vnet-jails-ec2/

DNS over HTTPS over Tor on OpenBSD.
https://pertho.net/2022/11/14/dohot/

Migrating to Syncthing from Nextcloud.
https://pertho.net/2024/07/08/migrating-to-syncthing/

FreeBSD Considering End of ppc64 Architecture Support.
https://talospace.com/2025/11/freebsd-considering-end-of-ppc64-support.html

FreeBSD Debates Sunsetting ppc64 Support.
https://osnews.com/story/144002/freebsd-debates-sunsetting-power64-power64le-support/

FreeBSD Foundation 2025 Q3 Status Update.
https://freebsdfoundation.org/blog/freebsd-foundation-q3-2025-status-update/

Unwrapping ZFS: Gifts from Open Source Community.
https://klarasystems.com/articles/zfs-community-contributions-2025/

HardenedBSD 14-STABLE Deprecation Notice: 2026/03.
https://groups.google.com/a/hardenedbsd.org/g/users/c/51IARO8noYo/m/UN5gVfq-CwAJ

Bhyve 2025-12-11 Production User Call.
https://youtube.com/watch?v=0oABGWodjLQ

OpenZFS 2025-12-10 Production User Call.
https://youtube.com/watch?v=2Idudwvlo6o

OPNSense 25.7.9 Released.
https://youtube.com/watch?v=hvn_0gWRIvI

Quick How to on Upgrading FreeBSD 14.3 to 15.0.
https://youtube.com/watch?v=a0UCttVQ19s

FreeBSD 15 on Raspberry PI 4/400 – Bene Factum.
https://youtube.com/watch?v=nvtq2_9AiIg

IVPN on NetBSD.
https://imil.net/blog/posts/2025/ivpn-on-netbsd/

XLibre FreeBSD Official.
https://daemondesktop.blogspot.com/2025/12/xlibre-freebsd-official.html

Hosting Static Blog on FreeBSD with Bastille Jails and Automated Deployment.
https://blog.hofstede.it/hosting-a-static-blog-on-freebsd-with-bastille-jails-and-automated-deployment/

Jenny Daily Drivers: Haiku R1/Beta5.
https://hackaday.com/2025/12/11/jennys-daily-drivers-haiku-r1-beta5/

My Talk About BoxyBSD at Credativ Virtualization Gathering 2025.
https://gyptazy.com/my-talk-about-boxybsd-at-credativs-virtualization-gathering-2025/

Rust Coreutils 0.5 Released – Inching Toward Full GNU Compatibility.
https://phoronix.com/news/Rust-Coreutils-0.5

BSD Now 641: OpenBSD to FreeBSD.
https://www.bsdnow.tv/641

Recovering Botched FreeBSD 14 Upgrade.
https://euroquis.nl/freebsd/2025/12/09/freebsd-upgrade.html

ZFS Storage Provisioning.
https://freebsd.uw.cz/2025/12/zfs-storage-provisioning.html

FreeBSD High Performance Network Stack Tuning Guide.
https://freebsd.uw.cz/2025/12/freebsd-high-performance-network-stack.html

My Two Months with OpenBSD.
https://youtube.com/watch?v=Cb2g2Deut8o

Hardware

USB Kill Shield.
https://usbkill.com/collections/usbkill-accessories/products/usbkill-shield

Intel Arc B580 vs. AMD Radeon RX 9000 vs. NVIDIA RTX 50 Series for llama.cpp Vulkan Performance.
https://phoronix.com/review/llama-cpp-vulkan-eoy2025/

How to Put Tailscale on Jailbroken Kindle.
https://tailscale.com/blog/tailscale-jailbroken-kindle

Samsung 1st Tri Fold Phone is Here.
https://extremetech.com/mobile/samsungs-first-tri-fold-phone-is-here-everything-you-need-to-know

Researcher Found Chinese KVM Had Undocumented Microphone and Communicated with China Based Servers.
https://tomshardware.com/tech-industry/cyber-security/researcher-finds-undocumented-microphone-and-major-security-flaws-in-sipeed-nanokvm

You Will Own NOTHING and Be Happy.
https://youtube.com/watch?v=S45rLuY48w4

AMD EPYC Embedded 2005 ZEN5 CPUs with Up to 16C and Up to 28 PCIe GEN5 Lanes and 10 Years Availability.
https://cnx-software.com/2025/12/11/up-to-16-core-amd-epyc-embedded-2005-series-zen-5-processors-support-up-28-pcie-gen5-lanes-offer-10-year-availability/

AMD Ryzen Embedded 8000 CPUs Target Industrial AI with 16 TOPS NPU.
https://cnx-software.com/2024/04/04/amd-ryzen-embedded-8000-processors-industrial-ai-16-tops-npu/

POWER11 Hits the Market This Month.
https://talospace.com/2025/07/power11-hits-market-this-month.html

How to Break Free from Smart TV Ads and Tracking.
https://arstechnica.com/gadgets/2025/12/the-ars-technica-guide-to-dumb-tvs/

Life

Nearly 60% of Last Year Graduates Not Landed 1st Job and 1/4 of Gen Z Workers Regret Going to College.
https://finance.yahoo.com/news/nearly-60-last-years-graduates-230106983.html

Gen Z Grads in U.K. Earning 30% Less than Millennials Did – New Data shows Degree Payoff is Collapsing.
https://finance.yahoo.com/news/u-k-grads-earning-30-155841076.html

Big Vitamin D Mistake.
https://pmc.ncbi.nlm.nih.gov/articles/PMC5541280/pdf/jpmph-50-4-278.pdf

Goodbye Microsoft: Schleswig-Holstein Relies on Open Source and Saves Millions.
https://heise.de/en/news/Goodbye-Microsoft-Schleswig-Holstein-relies-on-Open-Source-and-saves-millions-11105459.html

Why Smart People Get Hated in Stupid World.
https://youtube.com/watch?v=lrcVymdF2Ek

Homes of Hoarders.
https://slate.com/culture/2013/12/paula-salischiker-photographs-hoarders-in-britain-in-her-series-the-art-of-keeping.html

Other

Firefox 146.0 Now Available with Native Fractional Scaling on Wayland.
https://phoronix.com/news/Firefox-146-Released

Firefox 147.0 Beta Released with XDG Base Directory Support.
https://phoronix.com/news/Firefox-147-Beta

Humble Bundle Secretly Replaces Steam Keys for Epic Games Store Versions.
https://pcguide.com/news/gamers-warn-that-humble-bundle-is-secretly-replacing-steam-keys-for-epic-games-store-versions/

Humble Bundle Swaps Uour Unredeemed Steam Keys for Epic Versions w/o Any Notice.
https://gg.deals/gaming-news/humble-bundle-is-swapping-your-unredeemed-steam-keys-for-epic-versions-without-any-notice/

LibreWolf 146.0 Released.
https://codeberg.org/librewolf/bsys6/releases/tag/146.0-2

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Post by FreeBSD Newsflash via FreeBSD News Flash »

New committer: John Hall (src)
Top

Tracking kernel commits across branches

Post by Greg Kroah-Hartman via Linux Kernel Monkey Log »

With all of the different Linux kerenl stable releases happening (at least 1 stable branch and multiple longterm branches are active at any one point in time), keeping track of what commits are already applied to what branch, and what branch specific fixes should be applied to, can quickly get to be a very complex task if you attempt to do this manually. So I’ve created some tools to help make my life easier when doing the stable kerrnel maintenance work, which ended up making the work of tracking CVEs much simpler to manage in an automated way.

Top

As For Why We Left London

Post by Diego Elio Pettenò via Flameeyes's Weblog »

Nearly exactly five years ago, after my wife and I left our first flat in London, I wrote a post explaining the reasons we left. I thought it would be a good fit to explain why, five years later, we left London altogether and now effectively live in the suburbs.

So first of all, I have to say I don’t regret moving to London: my social life had definitely improved over there compared to Dublin (not least because I met my wife), I find myself a lot more comfortable in England than Ireland for a number of reasons (including but not limited to the slightly later store openings even here in the suburbs), and while my career at the previous employer suffered, I find myself in a fortunate position right now and I cannot really be complaining about that.

There are of course reasons why we needed to leave that specific apartment, and reasons why we decided to look outside of London — and the order of these coming up is… less than obvious. We decided we would be considering leaving London nearly two years ago, when my dayjob moved my office from a nice building a couple of minutes away from Tottenham Court Road, to a large building nearly 20 minutes away from the Piccadilly platform at Kings’ Cross. It might not sound like a significant change, but for living within London, that meant doubling my daily commute, had I stayed in-office — which meant I actually decided to try and “go remote”, to figure out if my displeasure in doing so during lockdown was due to the lack of office in-person time, or due to the lockdown, and the fear of COVID’s effect on diabetes. A year later I convinced myself the latter was the case, and we started looking outside of London — had I found that either my career or mental health were taking a hit by this situation, we would have been looking for a flat closer to the office.

Okay, but why outside of London? The answer is to be found in the current mess of a housing market that the UK in general, and London in particular, is in.

For starters, we completely excluded the idea of buying a flat like the ones we’ve been living in — we have a number of friends in the development we used to live in, and the way they appear to have been continually fighting with the company that is supposed to maintain and manage the development itself, which changed three times while I’ve been living there! Both FirstPort and Pod Management have been absolutely horrible to work with, both as a tenant (by first person experience), and as a landlord or leaseholder (second hand.) The worst part is that once you buy a flat, you’ll always be attached to the building and the company managing it, there’s no way out, and no real alternative short of the entire building agreeing to replace the company.

This wouldn’t be terrible if a building was all full of owners-occupiers: everybody living in the building has aligned incentives to pay as little as possible while ensuring the building is safe, usable, and good looking. But that’s effectively impossible in London, where domestic and foreign investors have been buying sometimes multiple units in the same building, for the rental income. Even without considering for short-term lets, the situation as it was in our old building was ridiculously skewed: the landlords have little-to-no incentive in maintaining the building operational – since even with the new renters rights bill, they can hide behind the property management company a fair bit – while the tenants have little-to-no incentive to ensure costs are kept reasonable — to give you a practical example, in our old building we had a communal garbage disposal area, and it was not uncommon for people to throw furniture and electronics in there; the property management company would often send reminders that the council fines the building every time this happens, and those fines end up raising the service charge bill, but that service charge is only paid by leaseholders, and landlords can’t revert it to the tenants. Even for leaseholders, when the provided alternative is to call a waste disposal company, or deliver the items to the council-managed waste disposal site, it is cheaper to fly-tip: a fine split between 50 people is still cheaper (to the individual) than a full waste disposal bill.

This actually felt like admitting defeat, for me — I have a personal belief that high density housing is the only solution we have, in the long term, as a society. But the current state of high density housing in London had me face many of the downsides of the setup, and eventually it broke me. And it didn’t really stop there: when we left London we spent a few months in a smaller flat in a building that was (supposedly) built to rent. This actually gave us a bit different perspective, and while having a single entity owning, managing, and leasing the units aligns the incentives a lot, it turns out that it wasn’t quite enough to make it worthwhile on a long-term basis. Though that might have been because of a quality issue with the building itself.

Speaking of personal defeats, moving to the suburbs means I’ll eventually have to give in, and get a driving license, despite my love for public transport in general. While our city provides some level of public transportation, from our estate to the centre there’s only one bus an hour during the day (busses are more frequent in the morning due to commuters and schools, but in general I don’t often go out during those times), and if you’re trying to go grocery shopping, or need to get somewhere at a specific time for an event, your only option is to use a car — either your own, or a ride-share.

In the meantime, I’ve resisted the call of four wheels by grabbing myself an electric trike (not a bike!), with cargo, so I can go shopping, grabbing a coffee, picking up or returning a parcel, and so so and so forth. This is only possible because, around Milton Keynes, we have a ton of shared pedestrian and cycle paths that are large enough for the trike (which is roughly as wide as a two-seater pram, not by chance since it is marketed to sit two children in it), making it safe enough for me to go around, despite being a less than confident cycler. The trike itself is worthy of a blog post by itself, since it’s not something that most people would think about, particularly here in the suburbs.

As an aside, much as I loved London’s public transport, and the significant improvement that the Elizabeth Line brought onto it, I have to admit that things were pretty much falling apart in the area we used to live. The 65 bus, which was our primary bus when living there and used to be fairly reliable, was shambolic when we left last year. One of the weekdays, just before we left, we had to wait for over an hour to take the 65 up to Ealing, which is longer than it would take to walk, and the only reason why we didn’t walk was that we had a parcel to send back that was too bulky to walk with!

One of the things we did expect to miss from London is the large amount of different things to do: events, concerts, shopping, food, and so on. Turns out this wasn’t as drastic as we thought it would be, either. Our new home is positioned almost exactly half-way through between London and Birmingham, making it quite possible to attend events on either city without extensive planning – we visited MegaCON Live in Birmingham earlier this year on a day trip, having decided to do so just two nights before – and for some of the venues (including, as it turns out, my own office) the trip is faster, though more expensive, than what we used to make from West London. There are also a number of events happening within the city itself, and we turned out to have a wider social circle in here than we used to have in London, for a number of different reasons.

Food selection was similarly a story that didn’t quite match our fear: while there is a smaller selection of venues in this city, the options that are available surprised our expectation significantly. And for those things that we can’t easily manage to find locally, we’re building up a list of small, independent companies that sell food in either ready-to-cook or kit form, including Matsudai Ramen (which we had the opportunity to try in Cardiff) and The Lincolnshire Smokehouse. So while it does take some time adjusting, life is not that difficult outside of the capital.

And for those wondering why we had to leave the flat we were living in, in the slight rush we did, when we did — the answer is that by February, we had spent four months without a working heating system. The building we lived in has a whole-building HVAC system, working both as “comfort cooling” in the summer, and heating in the winter, and it had been a bane of the residents for years. Some floors had no air conditioning for multiple summers, in a building that is all-glass all around, whose balconies reach the 60°C. And last summer, it was our turn, the air conditioning didn’t work for most of the summer, and in the worst heat wave in our memory we had to take a few days to just leave the house to avoid passing out from the heat. Eventually, they appeared to fix this, just in time for it not to be necessary at all — but I say appeared because it turned out that the refrigerant leak wasn’t actually found and fixed, and indeed by October there was none left for the HVAC to warm up the flats.

Why did it take over four months to deal with this? The first problem is that, despite identifying fairly quickly that the issue they were facing was a leak of refrigerant, they had no way to identify where such leak was. They suspected the valve on the compressor at first, and it wasn’t until November when they were sure the problem was in one of the flats… but couldn’t figure out which one without having access to every one of them. As it turns out, nobody in the designing phase of the building thought that it would be useful to have per-flat cut-off valves for the refrigerant. This is important not just for the diagnosing, as by January we knew which flat was affected (not ours), but we (together with other four flats) were held hostage of a contract dispute between the flat’s leaseholder and the property management company on who was supposed to pay for the repair. Not a good situation to be in.

This alone wouldn’t have been horrible if our landlord would have at least agreed to a reduction in rent, or paid for an alternative heating option, but while we attempted to negotiate that, he eventually decided he didn’t care for us to get to the end of our agreed lease, and let us leave early without having to pay for the early termination. For the record, the property management only offered to lend us an oil-filled heater — but nobody offered to pay for the additional cost of running it, compared to the existing heat-pump heating that we were supposed to have available.

So that’s the cliffs’ notes version of the reasons why we left London for a new built village in the largest city of Buckinghamshire, and are now enjoying a life that is not actually quieter, though definitely less stressful, at least for the time being.

Top

Microsoft Patch Tuesday, December 2025 Edition

Post by Brian Krebs via Krebs on Security »

Microsoft today pushed updates to fix at least 56 security flaws in its Windows operating systems and supported software. This final Patch Tuesday of 2025 tackles one zero-day bug that is already being exploited, as well as two publicly disclosed vulnerabilities.

Despite releasing a lower-than-normal number of security updates these past few months, Microsoft patched a whopping 1,129 vulnerabilities in 2025, an 11.9% increase from 2024. According to Satnam Narang at Tenable, this year marks the second consecutive year that Microsoft patched over one thousand vulnerabilities, and the third time it has done so since its inception.

The zero-day flaw patched today is CVE-2025-62221, a privilege escalation vulnerability affecting Windows 10 and later editions. The weakness resides in a component called the “Windows Cloud Files Mini Filter Driver” — a system driver that enables cloud applications to access file system functionalities.

“This is particularly concerning, as the mini filter is integral to services like OneDrive, Google Drive, and iCloud, and remains a core Windows component, even if none of those apps were installed,” said Adam Barnett, lead software engineer at Rapid7.

Only three of the flaws patched today earned Microsoft’s most-dire “critical” rating: Both CVE-2025-62554 and CVE-2025-62557 involve Microsoft Office, and both can exploited merely by viewing a booby-trapped email message in the Preview Pane. Another critical bug — CVE-2025-62562 — involves Microsoft Outlook, although Redmond says the Preview Pane is not an attack vector with this one.

But according to Microsoft, the vulnerabilities most likely to be exploited from this month’s patch batch are other (non-critical) privilege escalation bugs, including:

CVE-2025-62458 — Win32k
CVE-2025-62470 — Windows Common Log File System Driver
CVE-2025-62472 — Windows Remote Access Connection Manager
CVE-2025-59516 — Windows Storage VSP Driver
CVE-2025-59517 — Windows Storage VSP Driver

Kev Breen, senior director of threat research at Immersive, said privilege escalation flaws are observed in almost every incident involving host compromises.

“We don’t know why Microsoft has marked these specifically as more likely, but the majority of these components have historically been exploited in the wild or have enough technical detail on previous CVEs that it would be easier for threat actors to weaponize these,” Breen said. “Either way, while not actively being exploited, these should be patched sooner rather than later.”

One of the more interesting vulnerabilities patched this month is CVE-2025-64671, a remote code execution flaw in the Github Copilot Plugin for Jetbrains AI-based coding assistant that is used by Microsoft and GitHub. Breen said this flaw would allow attackers to execute arbitrary code by tricking the large language model (LLM) into running commands that bypass the user’s “auto-approve” settings.

CVE-2025-64671 is part of a broader, more systemic security crisis that security researcher Ari Marzuk has branded IDEsaster (IDE  stands for “integrated development environment”), which encompasses more than 30 separate vulnerabilities reported in nearly a dozen market-leading AI coding platforms, including Cursor, Windsurf, Gemini CLI, and Claude Code.

The other publicly-disclosed vulnerability patched today is CVE-2025-54100, a remote code execution bug in Windows Powershell on Windows Server 2008 and later that allows an unauthenticated attacker to run code in the security context of the user.

For anyone seeking a more granular breakdown of the security updates Microsoft pushed today, check out the roundup at the SANS Internet Storm Center. As always, please leave a note in the comments if you experience problems applying any of this month’s Windows patches.

Top

Drones to Diplomas: How Russia’s Largest Private University is Linked to a $25M Essay Mill

Post by Brian Krebs via Krebs on Security »

A sprawling academic cheating network turbocharged by Google Ads that has generated nearly $25 million in revenue has curious ties to a Kremlin-connected oligarch whose Russian university builds drones for Russia’s war against Ukraine.

The Nerdify homepage.

The link between essay mills and Russian attack drones might seem improbable, but understanding it begins with a simple question: How does a human-intensive academic cheating service stay relevant in an era when students can simply ask AI to write their term papers? The answer – recasting the business as an AI company – is just the latest chapter in a story of many rebrands that link the operation to Russia’s largest private university.

Search in Google for any terms related to academic cheating services — e.g., “help with exam online” or “term paper online” — and you’re likely to encounter websites with the words “nerd” or “geek” in them, such as thenerdify[.]com and geekly-hub[.]com. With a simple request sent via text message, you can hire their tutors to help with any assignment.

These nerdy and geeky-branded websites frequently cite their “honor code,” which emphasizes they do not condone academic cheating, will not write your term papers for you, and will only offer support and advice for customers. But according to This Isn’t Fine, a Substack blog about contract cheating and essay mills, the Nerdify brand of websites will happily ignore that mantra.

“We tested the quick SMS for a price quote,” wrote This Isn’t Fine author Joseph Thibault. “The honor code references and platitudes apparently stop at the website. Within three minutes, we confirmed that a full three-page, plagiarism- and AI-free MLA formatted Argumentative essay could be ours for the low price of $141.”

A screenshot from Joseph Thibault’s Substack post shows him purchasing a 3-page paper with the Nerdify service.

Google prohibits ads that “enable dishonest behavior.” Yet, a sprawling global essay and homework cheating network run under the Nerdy brands has quietly bought its way to the top of Google searches – booking revenues of almost $25 million through a maze of companies in Cyprus, Malta and Hong Kong, while pitching “tutoring” that delivers finished work that students can turn in.

When one Nerdy-related Google Ads account got shut down, the group behind the company would form a new entity with a front-person (typically a young Ukrainian woman), start a new ads account along with a new website and domain name (usually with “nerdy” in the brand), and resume running Google ads for the same set of keywords.

UK companies belonging to the group that have been shut down by Google Ads since Jan 2025 include:

Proglobal Solutions LTD (advertised nerdifyit[.]com);
AW Tech Limited (advertised thenerdify[.]com);
Geekly Solutions Ltd (advertised geekly-hub[.]com).

Currently active Google Ads accounts for the Nerdify brands include:

-OK Marketing LTD (advertising geekly-hub[.]net⁩), formed in the name of Olha Karpenko, a young Ukrainian woman;
Two Sigma Solutions LTD (advertising litero[.]ai), formed in the name of Olekszij (Alexey) Pokatilo.

Google’s Ads Transparency page for current Nerdify advertiser OK Marketing LTD.

Mr. Pokatilo has been in the essay-writing business since at least 2009, operating a paper-mill enterprise called Livingston Research alongside Alexander Korsukov, who is listed as an owner. According to a lengthy account from a former employee, Livingston Research mainly farmed its writing tasks out to low-cost workers from Kenya, Philippines, Pakistan, Russia and Ukraine.

Pokatilo moved from Ukraine to the United Kingdom in Sept. 2015 and co-founded a company called Awesome Technologies, which pitched itself as a way for people to outsource tasks by sending a text message to the service’s assistants.

The other co-founder of Awesome Technologies is 36-year-old Filip Perkon, a Swedish man living in London who touts himself as a serial entrepreneur and investor. Years before starting Awesome together, Perkon and Pokatilo co-founded a student group called Russian Business Week while the two were classmates at the London School of Economics. According to the Bulgarian investigative journalist Christo Grozev, Perkon’s birth certificate was issued by the Soviet Embassy in Sweden.

Alexey Pokatilo (left) and Filip Perkon at a Facebook event for startups in San Francisco in mid-2015.

Around the time Perkon and Pokatilo launched Awesome Technologies, Perkon was building a social media propaganda tool called the Russian Diplomatic Online Club, which Perkon said would “turbo-charge” Russian messaging online. The club’s newsletter urged subscribers to install in their Twitter accounts a third-party app called Tweetsquad that would retweet Kremlin messaging on the social media platform.

Perkon was praised by the Russian Embassy in London for his efforts: During the contentious Brexit vote that ultimately led to the United Kingdom leaving the European Union, the Russian embassy in London used this spam tweeting tool to auto-retweet the Russian ambassador’s posts from supporters’ accounts.

Neither Mr. Perkon nor Mr. Pokatilo replied to requests for comment.

A review of corporations tied to Mr. Perkon as indexed by the business research service North Data finds he holds or held director positions in several U.K. subsidiaries of Synergy University, Russia’s largest private education provider. Synergy has more than 35,000 students, and sells T-shirts with patriotic slogans such as “Crimea is Ours,” and “The Russian Empire — Reloaded.”

The president of Synergy University is Vadim Lobov, a Kremlin insider whose headquarters on the outskirts of Moscow reportedly features a wall-sized portrait of Russian President Vladimir Putin in the pop-art style of Andy Warhol. For a number of years, Lobov and Perkon co-produced a cross-cultural event in the U.K. called Russian Film Week.

Synergy President Vadim Lobov and Filip Perkon, speaking at a press conference for Russian Film Week, a cross-cultural event in the U.K. co-produced by both men.

Mr. Lobov was one of 11 individuals reportedly hand-picked by the convicted Russian spy Marina Butina to attend the 2017 National Prayer Breakfast held in Washington D.C. just two weeks after President Trump’s first inauguration.

While Synergy University promotes itself as Russia’s largest private educational institution, hundreds of international students tell a different story. Online reviews from students paint a picture of unkept promises: Prospective students from Nigeria, Kenya, Ghana, and other nations paying thousands in advance fees for promised study visas to Russia, only to have their applications denied with no refunds offered.

“My experience with Synergy University has been nothing short of heartbreaking,” reads one such account. “When I first discovered the school, their representative was extremely responsive and eager to assist. He communicated frequently and made me believe I was in safe hands. However, after paying my hard-earned tuition fees, my visa was denied. It’s been over 9 months since that denial, and despite their promises, I have received no refund whatsoever. My messages are now ignored, and the same representative who once replied instantly no longer responds at all. Synergy University, how can an institution in Europe feel comfortable exploiting the hopes of Africans who trust you with their life savings? This is not just unethical — it’s predatory.”

This pattern repeats across reviews by multilingual students from Pakistan, Nepal, India, and various African nations — all describing the same scheme: Attractive online marketing, promises of easy visa approval, upfront payment requirements, and then silence after visa denials.

Reddit discussions in r/Moscow and r/AskARussian are filled with warnings. “It’s a scam, a diploma mill,” writes one user. “They literally sell exams. There was an investigation on Rossiya-1 television showing students paying to pass tests.”

The Nerdify website’s “About Us” page says the company was co-founded by Pokatilo and an American named Brian Mellor. The latter identity seems to have been fabricated, or at least there is no evidence that a person with this name ever worked at Nerdify.

Rather, it appears that the SMS assistance company co-founded by Messrs. Pokatilo and Perkon (Awesome Technologies) fizzled out shortly after its creation, and that Nerdify soon adopted the process of accepting assignment requests via text message and routing them to freelance writers.

A closer look at an early “About Us” page for Nerdify in The Wayback Machine suggests that Mr. Perkon was the real co-founder of the company: The photo at the top of the page shows four people wearing Nerdify T-shirts seated around a table on a rooftop deck in San Francisco, and the man facing the camera is Perkon.

Filip Perkon, top right, is pictured wearing a Nerdify T-shirt in an archived copy of the company’s About Us page. Image: archive.org.

Where are they now? Pokatilo is currently running a startup called Litero.Ai, which appears to be an AI-based essay writing service. In July 2025, Mr. Pokatilo received pre-seed funding of $800,000 for Litero from an investment program backed by the venture capital firms AltaIR Capital, Yellow Rocks, Smart Partnership Capital, and I2BF Global Ventures.

Meanwhile, Filip Perkon is busy setting up toy rubber duck stores in Miami and in at least three locations in the United Kingdom. These “Duck World” shops market themselves as “the world’s largest duck store.”

This past week, Mr. Lobov was in India with Putin’s entourage on a charm tour with India’s Prime Minister Narendra Modi. Although Synergy is billed as an educational institution, a review of the company’s sprawling corporate footprint (via DNS) shows it also is assisting the Russian government in its war against Ukraine.

Synergy University President Vadim Lobov (right) pictured this week in India next to Natalia Popova, a Russian TV presenter known for her close ties to Putin’s family, particularly Putin’s daughter, who works with Popova at the education and culture-focused Innopraktika Foundation.

The website bpla.synergy[.]bot, for instance, says the company is involved in developing combat drones to aid Russian forces and to evade international sanctions on the supply and re-export of high-tech products.

A screenshot from the website of synergy,bot shows the company is actively engaged in building armed drones for the war in Ukraine.

KrebsOnSecurity would like to thank the anonymous researcher NatInfoSec for their assistance in this investigation.

Update, Dec. 8, 10:06 a.m. ET: Mr. Pokatilo responded to requests for comment after the publication of this story. Pokatilo said he has no relation to Synergy nor to Mr. Lobov, and that his work with Mr. Perkon ended with the dissolution of Awesome Technologies.

“I have had no involvement in any of his projects and business activities mentioned in the article and he has no involvement in Litero.ai,” Pokatilo said of Perkon.

Mr. Pokatilo said his new company Litero “does not provide contract cheating services and is built specifically to improve transparency and academic integrity in the age of universal use of AI by students.”

“I am Ukrainian,” he said in an email. “My close friends, colleagues, and some family members continue to live in Ukraine under the ongoing invasion. Any suggestion that I or my company may be connected in any way to Russia’s war efforts is deeply offensive on a personal level and harmful to the reputation of Litero.ai, a company where many team members are Ukrainian.”

Update, Dec. 11, 12:07 p.m. ET: Mr. Perkon responded to requests for comment after the publication of this story. Perkon said the photo of him in a Nerdify T-shirt (see screenshot above) was taken after a startup event in San Francisco, where he volunteered to act as a photo model to help friends with their project.

“I have no business or other relations to Nerdify or any other ventures in that space,” Mr. Perkon said in an email response. “As for Vadim Lobov, I worked for Venture Capital arm at Synergy until 2013 as well as his business school project in the UK, that didn’t get off the ground, so the company related to this was made dormant. Then Synergy kindly provided sponsorship for my Russian Film Week event that I created and ran until 2022 in the U.K., an event that became the biggest independent Russian film festival outside of Russia. Since the start of the Ukraine war in 2022 I closed the festival down.”

“I have had no business with Vadim Lobov since 2021 (the last film festival) and I don’t keep track of his endeavours,” Perkon continued. “As for Alexey Pokatilo, we are university friends. Our business relationship has ended after the concierge service Awesome Technologies didn’t work out, many years ago.”

Top

r730-01

Post by Dan Langille via Dan Langille's Other Diary »

This post has been replaced by a newer post.

This is the Dell R730 host known as r730-01.

For reference, the previous post on this server is still available.

This host has undergone major storage changes over the past few weeks. This post will reflect those changes

This is my primary developer server in my basement.

[16:52 r730-01 dvl ~] % uname -a
FreeBSD r730-01.int.unixathome.org 14.3-RELEASE-p5 FreeBSD 14.3-RELEASE-p5 GENERIC amd64

gpart

[16:54 r730-01 dvl ~] % gpart show
=>       40  488397088  nda4  GPT  (233G)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  freebsd-swap  (2.0G)
    4196352  484200448     3  freebsd-zfs  (231G)
  488396800        328        - free -  (164K)

=>        40  1953525088  nda6  GPT  (932G)
          40  1953525088     1  freebsd-zfs  (932G)

=>        40  1953525088  nda7  GPT  (932G)
          40  1953525088     1  freebsd-zfs  (932G)

=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

=>        40  1562824288  da0  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da1  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da3  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da2  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da5  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da4  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da7  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  1562824288  da6  GPT  (745G)
          40  1562824288    1  freebsd-zfs  (745G)

=>        40  7814037088  da11  GPT  (3.6T)
          40  7814037088     1  freebsd-zfs  (3.6T)

=>        40  7814037088  da10  GPT  (3.6T)
          40  7814037088     1  freebsd-zfs  (3.6T)

=>        40  7814037088  da9  GPT  (3.6T)
          40  7814037088    1  freebsd-zfs  (3.6T)

=>        40  7814037088  da8  GPT  (3.6T)
          40  7814037088    1  freebsd-zfs  (3.6T)

=>        40  7814037088  nda0  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda1  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda2  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda3  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  da12  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  da13  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda5  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda8  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

zpool list

[16:54 r730-01 dvl ~] % zpool list -v
NAME                                SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01                             5.81T  3.56G  5.81T        -         -     0%     0%  1.00x    ONLINE  -
  raidz2-0                         5.81T  3.56G  5.81T        -         -     0%  0.05%      -    ONLINE
    gpt/Y7P0A022TEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02ATEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02DTEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02GTEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02LTEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02MTEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A02QTEVE                745G      -      -        -         -      -      -      -    ONLINE
    gpt/Y7P0A033TEVE                745G      -      -        -         -      -      -      -    ONLINE
data02                              928G   649G   279G        -         -     7%    69%  1.00x    ONLINE  -
  mirror-0                          928G   649G   279G        -         -     7%  70.0%      -    ONLINE
    gpt/S6WSNJ0T208743F             932G      -      -        -         -      -      -      -    ONLINE
    gpt/S6WSNJ0T207774T             932G      -      -        -         -      -      -      -    ONLINE
data03                             7.25T  1.29T  5.96T        -         -    32%    17%  1.00x    ONLINE  -
  mirror-0                         3.62T   673G  2.97T        -         -    32%  18.1%      -    ONLINE
    gpt/WD_22492H800867            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/WD_230151801284            3.64T      -      -        -         -      -      -      -    ONLINE
  mirror-1                         3.62T   646G  2.99T        -         -    33%  17.4%      -    ONLINE
    gpt/WD_230151801478            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/WD_230151800473            3.64T      -      -        -         -      -      -      -    ONLINE
data04                             29.1T  4.49T  24.6T        -         -     0%    15%  1.00x    ONLINE  -
  raidz2-0                         29.1T  4.49T  24.6T        -         -     0%  15.4%      -    ONLINE
    gpt/S7KGNU0Y722875X            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S7KGNU0Y915666E            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S7KGNU0Y912937J            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S7KGNU0Y912955D            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S7U8NJ0Y716854P            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S7U8NJ0Y716801F            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S757NS0Y700758M            3.64T      -      -        -         -      -      -      -    ONLINE
    gpt/S757NS0Y700760R            3.64T      -      -        -         -      -      -      -    ONLINE
zroot                               107G  30.3G  76.7G        -         -    15%    28%  1.00x    ONLINE  -
  mirror-0                          107G  30.3G  76.7G        -         -    15%  28.3%      -    ONLINE
    gpt/zfs0_20170718AA0000185556   107G      -      -        -         -      -      -      -    ONLINE
    gpt/zfs1_20170719AA1178164201   107G      -      -        -         -      -      -      -    ONLINE

zpool status

[16:54 r730-01 dvl ~] % zpool status
  pool: data01
 state: ONLINE
  scan: scrub repaired 0B in 00:21:01 with 0 errors on Thu Dec  4 04:48:02 2025
config:

	NAME                  STATE     READ WRITE CKSUM
	data01                ONLINE       0     0     0
	  raidz2-0            ONLINE       0     0     0
	    gpt/Y7P0A022TEVE  ONLINE       0     0     0
	    gpt/Y7P0A02ATEVE  ONLINE       0     0     0
	    gpt/Y7P0A02DTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02GTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02LTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02MTEVE  ONLINE       0     0     0
	    gpt/Y7P0A02QTEVE  ONLINE       0     0     0
	    gpt/Y7P0A033TEVE  ONLINE       0     0     0

errors: No known data errors

  pool: data02
 state: ONLINE
  scan: scrub repaired 0B in 00:06:21 with 0 errors on Tue Dec  9 17:52:18 2025
config:

	NAME                     STATE     READ WRITE CKSUM
	data02                   ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    gpt/S6WSNJ0T208743F  ONLINE       0     0     0
	    gpt/S6WSNJ0T207774T  ONLINE       0     0     0

errors: No known data errors

  pool: data03
 state: ONLINE
  scan: scrub repaired 0B in 00:50:48 with 0 errors on Thu Dec  4 05:18:14 2025
config:

	NAME                     STATE     READ WRITE CKSUM
	data03                   ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    gpt/WD_22492H800867  ONLINE       0     0     0
	    gpt/WD_230151801284  ONLINE       0     0     0
	  mirror-1               ONLINE       0     0     0
	    gpt/WD_230151801478  ONLINE       0     0     0
	    gpt/WD_230151800473  ONLINE       0     0     0

errors: No known data errors

  pool: data04
 state: ONLINE
  scan: scrub repaired 0B in 00:35:09 with 0 errors on Wed Dec 10 01:06:08 2025
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
  scan: scrub repaired 0B in 00:01:12 with 0 errors on Mon Dec  8 23:52:58 2025
config:

	NAME                               STATE     READ WRITE CKSUM
	zroot                              ONLINE       0     0     0
	  mirror-0                         ONLINE       0     0     0
	    gpt/zfs0_20170718AA0000185556  ONLINE       0     0     0
	    gpt/zfs1_20170719AA1178164201  ONLINE       0     0     0

errors: No known data errors

zfs list

[16:55 r730-01 dvl ~] % zfs list
NAME                                                                 USED  AVAIL  REFER  MOUNTPOINT
data01                                                               293G  3.72T   205K  none
data01/jail_within_jail                                              617M  3.72T   205K  none
data01/jail_within_jail/jails                                        617M  3.72T   205K  none
data01/jail_within_jail/jails/freshports                             617M  3.72T   617M  none
data01/mkjail                                                       1.83G  3.72T   468M  /var/db/mkjail
data01/mkjail/14.3-RELEASE                                          1.37G  3.72T  1.37G  /var/db/mkjail/14.3-RELEASE
data01/reserved                                                      290G  4.00T   205K  none
data02                                                               829G  70.2G    96K  none
data02/freshports                                                    293G  70.2G    88K  none
data02/freshports/dev-ingress01                                      228G  70.2G    88K  none
data02/freshports/dev-ingress01/dvl-src                              197G  70.2G   197G  /jails/dev-ingress01/usr/home/dvl/src
data02/freshports/dev-ingress01/freshports                          21.8G  70.2G  1.54G  /jails/dev-ingress01/var/db/freshports
data02/freshports/dev-ingress01/freshports/cache                    1.22M  70.2G   132K  /jails/dev-ingress01/var/db/freshports/cache
data02/freshports/dev-ingress01/freshports/cache/html                896K  70.2G   360K  /jails/dev-ingress01/var/db/freshports/cache/html
data02/freshports/dev-ingress01/freshports/cache/spooling            224K  70.2G    96K  /jails/dev-ingress01/var/db/freshports/cache/spooling
data02/freshports/dev-ingress01/freshports/message-queues           20.1G  70.2G  20.3M  /jails/dev-ingress01/var/db/freshports/message-queues
data02/freshports/dev-ingress01/freshports/message-queues/archive   20.0G  70.2G  10.9G  /jails/dev-ingress01/var/db/freshports/message-queues/archive
data02/freshports/dev-ingress01/ingress                             5.68G  70.2G   124K  /jails/dev-ingress01/var/db/ingress
data02/freshports/dev-ingress01/ingress/latest_commits               588K  70.2G   108K  /jails/dev-ingress01/var/db/ingress/latest_commits
data02/freshports/dev-ingress01/ingress/message-queues              4.65M  70.2G  1.63M  /jails/dev-ingress01/var/db/ingress/message-queues
data02/freshports/dev-ingress01/ingress/repos                       5.67G  70.2G   120K  /jails/dev-ingress01/var/db/ingress/repos
data02/freshports/dev-ingress01/ingress/repos/doc                    673M  70.2G   668M  /jails/dev-ingress01/var/db/ingress/repos/doc
data02/freshports/dev-ingress01/ingress/repos/ports                 2.35G  70.2G  2.19G  /jails/dev-ingress01/var/db/ingress/repos/ports
data02/freshports/dev-ingress01/ingress/repos/src                   2.67G  70.2G  2.66G  /jails/dev-ingress01/var/db/ingress/repos/src
data02/freshports/dev-ingress01/jails                               3.22G  70.2G   104K  /jails/dev-ingress01/jails
data02/freshports/dev-ingress01/jails/freshports                    3.22G  70.2G   405M  /jails/dev-ingress01/jails/freshports
data02/freshports/dev-ingress01/jails/freshports/ports              2.82G  70.2G  2.51G  /jails/dev-ingress01/jails/freshports/usr/ports
data02/freshports/dev-ingress01/modules                             4.38M  70.2G  4.38M  /jails/dev-ingress01/usr/local/lib/perl5/site_perl/FreshPorts
data02/freshports/dev-ingress01/scripts                             3.27M  70.2G  3.27M  /jails/dev-ingress01/usr/local/libexec/freshports
data02/freshports/dev-nginx01                                       54.6M  70.2G    96K  none
data02/freshports/dev-nginx01/www                                   54.5M  70.2G    96K  /jails/dev-nginx01/usr/local/www
data02/freshports/dev-nginx01/www/freshports                        51.7M  70.2G  51.7M  /jails/dev-nginx01/usr/local/www/freshports
data02/freshports/dev-nginx01/www/freshsource                       2.71M  70.2G  2.71M  /jails/dev-nginx01/usr/local/www/freshsource
data02/freshports/dvl-ingress01                                     17.8G  70.2G    96K  none
data02/freshports/dvl-ingress01/dvl-src                             80.1M  70.2G  80.1M  /jails/dvl-ingress01/usr/home/dvl/src
data02/freshports/dvl-ingress01/freshports                          3.80G  70.2G    96K  /jails/dvl-ingress01/var/db/freshports
data02/freshports/dvl-ingress01/freshports/cache                    1.25M  70.2G    96K  /jails/dvl-ingress01/var/db/freshports/cache
data02/freshports/dvl-ingress01/freshports/cache/html                948K  70.2G   332K  /jails/dvl-ingress01/var/db/freshports/cache/html
data02/freshports/dvl-ingress01/freshports/cache/spooling            232K  70.2G   104K  /jails/dvl-ingress01/var/db/freshports/cache/spooling
data02/freshports/dvl-ingress01/freshports/message-queues           3.80G  70.2G  18.1M  /jails/dvl-ingress01/var/db/freshports/message-queues
data02/freshports/dvl-ingress01/freshports/message-queues/archive   3.77G  70.2G  3.76G  /jails/dvl-ingress01/var/db/freshports/message-queues/archive
data02/freshports/dvl-ingress01/ingress                             8.50G  70.2G   120K  /jails/dvl-ingress01/var/db/ingress
data02/freshports/dvl-ingress01/ingress/latest_commits               220K  70.2G   100K  /jails/dvl-ingress01/var/db/ingress/latest_commits
data02/freshports/dvl-ingress01/ingress/message-queues              4.83M  70.2G  2.17M  /jails/dvl-ingress01/var/db/ingress/message-queues
data02/freshports/dvl-ingress01/ingress/repos                       8.49G  70.2G   112K  /jails/dvl-ingress01/var/db/ingress/repos
data02/freshports/dvl-ingress01/ingress/repos/doc                    697M  70.2G   667M  /jails/dvl-ingress01/var/db/ingress/repos/doc
data02/freshports/dvl-ingress01/ingress/repos/ports                 3.45G  70.2G  2.23G  /jails/dvl-ingress01/var/db/ingress/repos/ports
data02/freshports/dvl-ingress01/ingress/repos/src                   4.36G  70.2G  2.68G  /jails/dvl-ingress01/var/db/ingress/repos/src
data02/freshports/dvl-ingress01/jails                               5.41G  70.2G   104K  /jails/dvl-ingress01/jails
data02/freshports/dvl-ingress01/jails/freshports                    5.41G  70.2G   404M  /jails/dvl-ingress01/jails/freshports
data02/freshports/dvl-ingress01/jails/freshports/ports              5.01G  70.2G  2.53G  /jails/dvl-ingress01/jails/freshports/usr/ports
data02/freshports/dvl-ingress01/modules                             2.67M  70.2G  2.67M  /jails/dvl-ingress01/usr/local/lib/perl5/site_perl/FreshPorts
data02/freshports/dvl-ingress01/scripts                             2.32M  70.2G  2.32M  /jails/dvl-ingress01/usr/local/libexec/freshports
data02/freshports/dvl-nginx01                                       22.2M  70.2G    96K  none
data02/freshports/dvl-nginx01/www                                   22.1M  70.2G    96K  none
data02/freshports/dvl-nginx01/www/freshports                        20.2M  70.2G  20.2M  /jails/dvl-nginx01/usr/local/www/freshports
data02/freshports/dvl-nginx01/www/freshsource                       1.78M  70.2G  1.78M  /jails/dvl-nginx01/usr/local/www/freshsource
data02/freshports/jailed                                             876M  70.2G    96K  none
data02/freshports/jailed/dev-ingress01                                96K  70.2G    96K  none
data02/freshports/jailed/dev-nginx01                                 386M  70.2G    96K  none
data02/freshports/jailed/dev-nginx01/cache                           386M  70.2G    96K  /var/db/freshports/cache
data02/freshports/jailed/dev-nginx01/cache/categories               2.87M  70.2G  1.73M  /var/db/freshports/cache/categories
data02/freshports/jailed/dev-nginx01/cache/commits                    96K  70.2G    96K  /var/db/freshports/cache/commits
data02/freshports/jailed/dev-nginx01/cache/daily                    1.61M  70.2G  1.37M  /var/db/freshports/cache/daily
data02/freshports/jailed/dev-nginx01/cache/general                  1020K  70.2G   588K  /var/db/freshports/cache/general
data02/freshports/jailed/dev-nginx01/cache/news                      444K  70.2G   144K  /var/db/freshports/cache/news
data02/freshports/jailed/dev-nginx01/cache/packages                 40.3M  70.2G  38.8M  /var/db/freshports/cache/packages
data02/freshports/jailed/dev-nginx01/cache/pages                      96K  70.2G    96K  /var/db/freshports/cache/pages
data02/freshports/jailed/dev-nginx01/cache/ports                     339M  70.2G   317M  /var/db/freshports/cache/ports
data02/freshports/jailed/dev-nginx01/cache/spooling                  376K  70.2G   120K  /var/db/freshports/cache/spooling
data02/freshports/jailed/dvl-ingress01                               192K  70.2G    96K  none
data02/freshports/jailed/dvl-ingress01/distfiles                      96K  70.2G    96K  none
data02/freshports/jailed/dvl-nginx01                                1.84M  70.2G    96K  none
data02/freshports/jailed/dvl-nginx01/cache                          1.66M  70.2G   148K  /var/db/freshports/cache
data02/freshports/jailed/dvl-nginx01/cache/categories                 96K  70.2G    96K  /var/db/freshports/cache/categories
data02/freshports/jailed/dvl-nginx01/cache/commits                    96K  70.2G    96K  /var/db/freshports/cache/commits
data02/freshports/jailed/dvl-nginx01/cache/daily                      96K  70.2G    96K  /var/db/freshports/cache/daily
data02/freshports/jailed/dvl-nginx01/cache/general                    96K  70.2G    96K  /var/db/freshports/cache/general
data02/freshports/jailed/dvl-nginx01/cache/news                      428K  70.2G   144K  /var/db/freshports/cache/news
data02/freshports/jailed/dvl-nginx01/cache/packages                   96K  70.2G    96K  /var/db/freshports/cache/packages
data02/freshports/jailed/dvl-nginx01/cache/pages                      96K  70.2G    96K  /var/db/freshports/cache/pages
data02/freshports/jailed/dvl-nginx01/cache/ports                      96K  70.2G    96K  /var/db/freshports/cache/ports
data02/freshports/jailed/dvl-nginx01/cache/spooling                  352K  70.2G   120K  /var/db/freshports/cache/spooling
data02/freshports/jailed/dvl-nginx01/freshports                       96K  70.2G    96K  none
data02/freshports/jailed/stage-ingress01                             192K  70.2G    96K  none
data02/freshports/jailed/stage-ingress01/data                         96K  70.2G    96K  none
data02/freshports/jailed/stage-nginx01                               413M  70.2G    96K  none
data02/freshports/jailed/stage-nginx01/cache                         413M  70.2G   248K  /var/db/freshports/cache
data02/freshports/jailed/stage-nginx01/cache/categories             2.46M  70.2G  1.73M  /var/db/freshports/cache/categories
data02/freshports/jailed/stage-nginx01/cache/commits                  96K  70.2G    96K  /var/db/freshports/cache/commits
data02/freshports/jailed/stage-nginx01/cache/daily                  1004K  70.2G   772K  /var/db/freshports/cache/daily
data02/freshports/jailed/stage-nginx01/cache/general                 848K  70.2G   488K  /var/db/freshports/cache/general
data02/freshports/jailed/stage-nginx01/cache/news                    436K  70.2G   144K  /var/db/freshports/cache/news
data02/freshports/jailed/stage-nginx01/cache/packages               23.8M  70.2G  22.6M  /var/db/freshports/cache/packages
data02/freshports/jailed/stage-nginx01/cache/pages                    96K  70.2G    96K  /var/db/freshports/cache/pages
data02/freshports/jailed/stage-nginx01/cache/ports                   384M  70.2G   344M  /var/db/freshports/cache/ports
data02/freshports/jailed/stage-nginx01/cache/spooling                384K  70.2G   120K  /var/db/freshports/cache/spooling
data02/freshports/jailed/test-ingress01                              192K  70.2G    96K  none
data02/freshports/jailed/test-ingress01/data                          96K  70.2G    96K  none
data02/freshports/jailed/test-nginx01                               74.0M  70.2G    96K  none
data02/freshports/jailed/test-nginx01/cache                         73.9M  70.2G   236K  /var/db/freshports/cache
data02/freshports/jailed/test-nginx01/cache/categories              1.01M  70.2G   628K  /var/db/freshports/cache/categories
data02/freshports/jailed/test-nginx01/cache/commits                   96K  70.2G    96K  /var/db/freshports/cache/commits
data02/freshports/jailed/test-nginx01/cache/daily                    352K  70.2G   280K  /var/db/freshports/cache/daily
data02/freshports/jailed/test-nginx01/cache/general                  932K  70.2G   524K  /var/db/freshports/cache/general
data02/freshports/jailed/test-nginx01/cache/news                     444K  70.2G   144K  /var/db/freshports/cache/news
data02/freshports/jailed/test-nginx01/cache/packages                8.16M  70.2G  7.25M  /var/db/freshports/cache/packages
data02/freshports/jailed/test-nginx01/cache/pages                     96K  70.2G    96K  /var/db/freshports/cache/pages
data02/freshports/jailed/test-nginx01/cache/ports                   62.3M  70.2G  55.9M  /var/db/freshports/cache/ports
data02/freshports/jailed/test-nginx01/cache/spooling                 384K  70.2G   120K  /var/db/freshports/cache/spooling
data02/freshports/stage-ingress01                                   20.7G  70.2G    96K  none
data02/freshports/stage-ingress01/cache                             1.22M  70.2G    96K  /jails/stage-ingress01/var/db/freshports/cache
data02/freshports/stage-ingress01/cache/html                         940K  70.2G   316K  /jails/stage-ingress01/var/db/freshports/cache/html
data02/freshports/stage-ingress01/cache/spooling                     216K  70.2G    96K  /jails/stage-ingress01/var/db/freshports/cache/spooling
data02/freshports/stage-ingress01/freshports                        10.4G  70.2G    96K  none
data02/freshports/stage-ingress01/freshports/archive                10.3G  70.2G  10.3G  /jails/stage-ingress01/var/db/freshports/message-queues/archive
data02/freshports/stage-ingress01/freshports/message-queues         40.2M  70.2G  16.6M  /jails/stage-ingress01/var/db/freshports/message-queues
data02/freshports/stage-ingress01/ingress                           5.37G  70.2G    96K  /jails/stage-ingress01/var/db/ingress
data02/freshports/stage-ingress01/ingress/latest_commits             524K  70.2G   100K  /jails/stage-ingress01/var/db/ingress/latest_commits
data02/freshports/stage-ingress01/ingress/message-queues            4.33M  70.2G  1.18M  /jails/stage-ingress01/var/db/ingress/message-queues
data02/freshports/stage-ingress01/ingress/repos                     5.36G  70.2G  5.21G  /jails/stage-ingress01/var/db/ingress/repos
data02/freshports/stage-ingress01/jails                              405M  70.2G   104K  /jails/stage-ingress01/jails
data02/freshports/stage-ingress01/jails/freshports                   405M  70.2G   404M  /jails/stage-ingress01/jails/freshports
data02/freshports/stage-ingress01/ports                             4.54G  70.2G  2.52G  /jails/stage-ingress01/jails/freshports/usr/ports
data02/freshports/test-ingress01                                    25.2G  70.2G    96K  none
data02/freshports/test-ingress01/freshports                         12.1G  70.2G  1.54G  /jails/test-ingress01/var/db/freshports
data02/freshports/test-ingress01/freshports/cache                   1.18M  70.2G    96K  /jails/test-ingress01/var/db/freshports/cache
data02/freshports/test-ingress01/freshports/cache/html               900K  70.2G   348K  /jails/test-ingress01/var/db/freshports/cache/html
data02/freshports/test-ingress01/freshports/cache/spooling           216K  70.2G    96K  /jails/test-ingress01/var/db/freshports/cache/spooling
data02/freshports/test-ingress01/freshports/message-queues          10.4G  70.2G  17.7M  /jails/test-ingress01/var/db/freshports/message-queues
data02/freshports/test-ingress01/freshports/message-queues/archive  10.4G  70.2G  10.4G  /jails/test-ingress01/var/db/freshports/message-queues/archive
data02/freshports/test-ingress01/ingress                            8.11G  70.2G   120K  /jails/test-ingress01/var/db/ingress
data02/freshports/test-ingress01/ingress/latest_commits              464K  70.2G   100K  /jails/test-ingress01/var/db/ingress/latest_commits
data02/freshports/test-ingress01/ingress/message-queues             4.18M  70.2G  1.17M  /jails/test-ingress01/var/db/ingress/message-queues
data02/freshports/test-ingress01/ingress/repos                      8.11G  70.2G  5.32G  /jails/test-ingress01/var/db/ingress/repos
data02/freshports/test-ingress01/jails                              4.91G  70.2G    96K  /jails/test-ingress01/jails
data02/freshports/test-ingress01/jails/freshports                   4.91G  70.2G   405M  /jails/test-ingress01/jails/freshports
data02/freshports/test-ingress01/jails/freshports/ports             4.51G  70.2G  2.49G  /jails/test-ingress01/jails/freshports/usr/ports
data02/jails                                                         287G  70.2G  9.54G  /jails
data02/jails/bacula                                                 16.5G  70.2G  16.1G  /jails/bacula
data02/jails/bacula-sd-02                                           3.10G  70.2G  2.62G  /jails/bacula-sd-02
data02/jails/bacula-sd-03                                           4.36G  70.2G  3.88G  /jails/bacula-sd-03
data02/jails/besser                                                 4.97G  70.2G  3.82G  /jails/besser
data02/jails/certs                                                  2.69G  70.2G  2.21G  /jails/certs
data02/jails/certs-rsync                                            2.67G  70.2G  2.20G  /jails/certs-rsync
data02/jails/cliff2                                                 2.71G  70.2G  2.24G  /jails/cliff2
data02/jails/dev-ingress01                                          4.07G  70.2G  3.25G  /jails/dev-ingress01
data02/jails/dev-nginx01                                            3.94G  70.2G  3.14G  /jails/dev-nginx01
data02/jails/dev-pgeu                                               6.86G  70.2G  6.86G  /jails/dev-pgeu
data02/jails/dns-hidden-master                                      2.84G  70.2G  2.33G  /jails/dns-hidden-master
data02/jails/dns1                                                   10.2G  70.2G  4.54G  /jails/dns1
data02/jails/dvl-ingress01                                          6.99G  70.2G  4.95G  /jails/dvl-ingress01
data02/jails/dvl-nginx01                                            1.80G  70.2G  1.02G  /jails/dvl-nginx01
data02/jails/freebsd143                                              501M  70.2G   501M  /jails/freebsd143
data02/jails/git                                                    4.42G  70.2G  3.94G  /jails/git
data02/jails/jail_within_jail                                       1.03G  70.2G   618M  /jails/jail_within_jail
data02/jails/mqtt01                                                 3.20G  70.2G  2.72G  /jails/mqtt01
data02/jails/mydev                                                  25.0G  70.2G  21.4G  /jails/mydev
data02/jails/mysql01                                                14.0G  70.2G  7.59G  /jails/mysql01
data02/jails/nsnotify                                               2.92G  70.2G  2.38G  /jails/nsnotify
data02/jails/pg01                                                   45.3G  70.2G  10.0G  /jails/pg01
data02/jails/pg02                                                   10.4G  70.2G  9.90G  /jails/pg02
data02/jails/pg03                                                   10.2G  70.2G  8.86G  /jails/pg03
data02/jails/pkg01                                                  15.8G  70.2G  13.0G  /jails/pkg01
data02/jails/samdrucker                                             4.37G  70.2G  3.89G  /jails/samdrucker
data02/jails/serpico                                                2.76G  70.2G  2.27G  /jails/serpico
data02/jails/stage-ingress01                                        4.35G  70.2G  2.49G  /jails/stage-ingress01
data02/jails/stage-nginx01                                          2.07G  70.2G  1.21G  /jails/stage-nginx01
data02/jails/svn                                                    10.1G  70.2G  9.51G  /jails/svn
data02/jails/talos                                                  2.65G  70.2G  2.19G  /jails/talos
data02/jails/test-ingress01                                         1.92G  70.2G  1.17G  /jails/test-ingress01
data02/jails/test-nginx01                                           2.03G  70.2G  1.14G  /jails/test-nginx01
data02/jails/unifi01                                                29.1G  70.2G  11.5G  /jails/unifi01
data02/jails/webserver                                              12.0G  70.2G  11.0G  /jails/webserver
data02/reserved                                                      180G   250G    96K  none
data02/vm                                                           68.6G  70.2G  6.41G  /usr/local/vm
data02/vm/freebsd-test                                               701M  70.2G   112K  /usr/local/vm/freebsd-test
data02/vm/freebsd-test/disk0                                         700M  70.2G   700M  -
data02/vm/hass                                                      58.6G  70.2G  14.3G  /usr/local/vm/hass
data02/vm/home-assistant                                             351M  70.2G   351M  /usr/local/vm/home-assistant
data02/vm/myguest                                                   2.55G  70.2G  2.55G  /usr/local/vm/myguest
data03                                                              1.64T  5.48T    96K  none
data03/acme-certs                                                   11.9M  5.48T    96K  none
data03/acme-certs/certs                                             6.50M  5.48T  5.67M  /jails/certs/var/db/acme/certs
data03/acme-certs/certs-for-rsync                                   5.26M  5.48T  4.03M  /jails/certs/var/db/certs-for-rsync
data03/dvl                                                           120K  5.48T   120K  /jails/dev-ingress01/usr/home/dvl/src
data03/jail_within_jail                                             1.08G  5.48T    96K  none
data03/jail_within_jail/jails                                       1.08G  5.48T    96K  none
data03/jail_within_jail/jails/freshports                            1.08G  5.48T  1.08G  none
data03/librenms-rrd                                                  117G  5.48T  25.5G  /jails/besser/var/db/librenms/rrd
data03/pg01                                                         71.2G  5.48T    96K  none
data03/pg01/freshports.dvl                                          37.1G  5.48T  27.6G  /jails/pg01/var/db/postgres.freshports.dvl
data03/pg01/postgres                                                34.1G  5.48T  27.6G  /jails/pg01/var/db/postgres
data03/pg02                                                         78.5G  5.48T    88K  none
data03/pg02/postgres                                                78.5G  5.48T  51.8G  /jails/pg02/var/db/postgres
data03/pg02/rsyncer                                                 1.02M  5.48T   136K  /jails/pg02/usr/home/rsyncer/backups
data03/pg03                                                          772G  5.48T    88K  none
data03/pg03/postgres                                                 573G  5.48T   550G  /jails/pg03/var/db/postgres
data03/pg03/rsyncer                                                  199G  5.48T  33.2G  /jails/pg03/usr/home/rsyncer/backups
data03/poudriere                                                     244G  5.48T    96K  /usr/local/poudriere
data03/poudriere/ccache                                             16.5M  5.48T   104K  /var/ccache
data03/poudriere/ccache/ccache.13amd64                                96K  5.48T    96K  /var/ccache/ccache.13amd64
data03/poudriere/ccache/ccache.amd64                                16.3M  5.48T  16.3M  /var/ccache/ccache.amd64
data03/poudriere/data                                               92.5G  5.48T  28.1G  /usr/local/poudriere/data
data03/poudriere/data/cache                                          206M  5.48T   174M  /usr/local/poudriere/data/cache
data03/poudriere/data/cronjob-logs                                  4.01M  5.48T  1.59M  /usr/local/poudriere/data/cronjob-logs
data03/poudriere/data/packages                                      64.1G  5.48T  58.9G  /usr/local/poudriere/data/packages
data03/poudriere/distfiles                                           132G  5.48T   132G  /usr/ports/distfiles
data03/poudriere/jails                                              2.75G  5.48T    88K  /usr/local/poudriere/jails
data03/poudriere/jails/142amd64                                     1.37G  5.48T  1.37G  /usr/local/poudriere/jails/142amd64
data03/poudriere/jails/143amd64                                     1.38G  5.48T  1.38G  /usr/local/poudriere/jails/143amd64
data03/poudriere/ports                                              16.4G  5.48T  3.25G  /usr/local/poudriere/ports
data03/poudriere/ports/default                                      2.97G  5.48T  2.78G  /usr/local/poudriere/ports/default
data03/poudriere/ports/main                                         1.60G  5.48T  1.60G  /usr/local/poudriere/ports/main
data03/poudriere/ports/pgeu_system                                  1.06G  5.48T  1.06G  /usr/local/poudriere/ports/pgeu_system
data03/poudriere/ports/testing                                      7.02G  5.48T  2.71G  /usr/local/poudriere/ports/testing
data03/poudriere/test                                                 96K  5.48T    96K  /usr/local/poudriere/test
data03/public                                                       23.4G  5.48T  23.4G  none
data03/repos                                                        9.05G  5.48T    88K  none
data03/repos/gitea                                                   811M  5.48T   307M  /jails/git/var/db/gitea
data03/repos/subversion                                             8.25G  5.48T  7.75G  /jails/svn/usr/local/svn/repos
data03/reserved                                                      362G  5.84T    96K  none
data04                                                              3.19T  17.4T   205K  /data04
data04/bacula                                                       3.19T  17.4T   205K  /jails/bacula-sd-03/usr/local/bacula
data04/bacula/volumes                                               3.19T  17.4T   205K  /jails/bacula-sd-03/usr/local/bacula/volumes
data04/bacula/volumes/DiffFile-03                                    405G  17.4T   405G  /jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03
data04/bacula/volumes/FullFile-03                                   2.45T  17.4T  2.44T  /jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03
data04/bacula/volumes/IncrFile-03                                    360G  17.4T   360G  /jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03
data04/bacula/working                                                367K  17.4T   247K  /jails/bacula-sd-03/usr/local/bacula/working
zroot                                                               52.3G  51.4G    96K  /zroot
zroot/ROOT                                                          12.6G  51.4G    96K  none
zroot/ROOT/14.3-RELEASE-p5_2025-11-29_152045                           8K  51.4G  11.2G  /
zroot/ROOT/default                                                  12.6G  51.4G  11.2G  /
zroot/reserved                                                      22.0G  73.4G    96K  /zroot/reserved
zroot/tmp                                                           33.9M  51.4G   136K  /tmp
zroot/usr                                                           15.6G  51.4G    96K  /usr
zroot/usr/home                                                      15.6G  51.4G  15.6G  /usr/home
zroot/usr/ports                                                       96K  51.4G    96K  /usr/ports
zroot/usr/src                                                         96K  51.4G    96K  /usr/src
zroot/var                                                           28.6M  51.4G    96K  /var
zroot/var/audit                                                       96K  51.4G    96K  /var/audit
zroot/var/crash                                                       96K  51.4G    96K  /var/crash
zroot/var/log                                                       23.5M  51.4G  6.17M  /var/log
zroot/var/mail                                                       216K  51.4G   104K  /var/mail
zroot/var/tmp                                                       4.66M  51.4G  2.39M  /var/tmp

dmesg

[16:55 r730-01 dvl ~] % dmesg
---<<BOOT>>---
Copyright (c) 1992-2023 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.3-RELEASE-p5 GENERIC amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
VT(efifb): resolution 1024x768
CPU microcode: no matching update found
CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz (2300.08-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306f2  Family=0x6  Model=0x3f  Stepping=2
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x37ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,NFPUSG>
  Structured Extended Features3=0x9c000400<MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 412312666112 (393212 MB)
avail memory = 401501585408 (382901 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 72 CPUs
FreeBSD/SMP: 2 package(s) x 18 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23
ioapic1 <Version 2.0> irqs 24-47
ioapic2 <Version 2.0> irqs 48-71
Launching APs: 1 24 59 17 68 45 38 63 6 3 40 42 36 15 5 41 49 33 34 10 52 11 14 25 51 9 44 56 70 8 29 53 62 13 12 64 54 7 47 46 61 50 35 71 60 28 23 32 65 16 18 66 20 48 67 30 4 55 21 69 22 43 2 31 37 58 26 39 19 27 57
random: entropy device external interface
kbd0 at kbdmux0
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
smbios0: <System Management BIOS> at iomem 0x7af0a000-0x7af0a01e
smbios0: Entry point: v2.1 (32-bit), Version: 2.8, BCD Revision: 2.8
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <DELL PE_SC3>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Event timer "HPET4" frequency 14318180 Hz quality 340
Event timer "HPET5" frequency 14318180 Hz quality 340
Event timer "HPET6" frequency 14318180 Hz quality 340
Event timer "HPET7" frequency 14318180 Hz quality 340
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <dasp, performance counters> at device 8.2 (no driver attached)
pci0: <dasp, performance counters> at device 9.2 (no driver attached)
pci0: <dasp, performance counters> at device 11.1 (no driver attached)
pci0: <dasp, performance counters> at device 11.2 (no driver attached)
pci0: <dasp, performance counters> at device 16.1 (no driver attached)
pci0: <dasp, performance counters> at device 16.6 (no driver attached)
pci0: <dasp, performance counters> at device 18.1 (no driver attached)
pci0: <dasp, performance counters> at device 18.5 (no driver attached)
pcib1: <ACPI Host-PCI bridge> on acpi0
pci1: <ACPI PCI bus> on pcib1
pci1: <dasp, performance counters> at device 8.2 (no driver attached)
pci1: <dasp, performance counters> at device 9.2 (no driver attached)
pci1: <dasp, performance counters> at device 11.1 (no driver attached)
pci1: <dasp, performance counters> at device 11.2 (no driver attached)
pci1: <dasp, performance counters> at device 16.1 (no driver attached)
pci1: <dasp, performance counters> at device 16.6 (no driver attached)
pci1: <dasp, performance counters> at device 18.1 (no driver attached)
pci1: <dasp, performance counters> at device 18.5 (no driver attached)
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> on acpi0
apei0: <ACPI Platform Error Interface> on acpi0
pcib2: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0
pci2: <ACPI PCI bus> numa-domain 0 on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 0 on pci2
pci3: <ACPI PCI bus> numa-domain 0 on pcib3
AVAGO MegaRAID SAS FreeBSD mrsas driver version: 07.709.04.00-fbsd
mrsas0: <AVAGO Invader SAS Controller> port 0x2000-0x20ff mem 0x92500000-0x9250ffff,0x92400000-0x924fffff at device 0.0 numa-domain 0 on pci3
mrsas0: FW now in Ready state
mrsas0: Using MSI-X with 72 number of vectors
mrsas0: FW supports <96> MSIX vector,Online CPU 72 Current MSIX <72>
mrsas0: mrsas_init_adapter: sc->reply_q_depth 0x740,sc->request_alloc_sz 0x1cf8, sc->reply_alloc_sz 0x3a00,sc->io_frames_alloc_sz 0x3a100
mrsas0: max sge: 0x46, max chain frame size: 0x400, max fw cmd: 0x39f sc->chain_frames_alloc_sz: 0xe7c00
mrsas0: Issuing IOC INIT command to FW.
mrsas0: IOC INIT response received from FW.
mrsas0: System PD created target ID: 0x0
mrsas0: System PD created target ID: 0x1
mrsas0: System PD created target ID: 0x2
mrsas0: System PD created target ID: 0x3
mrsas0: System PD created target ID: 0x4
mrsas0: System PD created target ID: 0x5
mrsas0: System PD created target ID: 0x6
mrsas0: System PD created target ID: 0x7
mrsas0: System PD created target ID: 0x8
mrsas0: System PD created target ID: 0x9
mrsas0: System PD created target ID: 0xa
mrsas0: System PD created target ID: 0xb
mrsas0: System PD created target ID: 0xc
mrsas0: System PD created target ID: 0xd
mrsas0: FW supports: UnevenSpanSupport=1

mrsas0: max_fw_cmds: 927  max_scsi_cmds: 911
mrsas0: MSI-x interrupts setup success
mrsas0: mrsas_ocr_thread
pcib4: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 0 on pci2
pci4: <ACPI PCI bus> numa-domain 0 on pcib4
nvme0: <Generic NVMe Device> mem 0x92300000-0x92303fff at device 0.0 numa-domain 0 on pci4
pcib5: <ACPI PCI-PCI bridge> at device 2.1 numa-domain 0 on pci2
pci5: <ACPI PCI bus> numa-domain 0 on pcib5
nvme1: <Generic NVMe Device> mem 0x92200000-0x92203fff at device 0.0 numa-domain 0 on pci5
pcib6: <ACPI PCI-PCI bridge> at device 2.2 numa-domain 0 on pci2
pci6: <ACPI PCI bus> numa-domain 0 on pcib6
nvme2: <Generic NVMe Device> mem 0x92100000-0x92103fff at device 0.0 numa-domain 0 on pci6
pcib7: <ACPI PCI-PCI bridge> at device 2.3 numa-domain 0 on pci2
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
nvme3: <Generic NVMe Device> mem 0x92000000-0x92003fff at device 0.0 numa-domain 0 on pci7
pcib8: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 0 on pci2
pci8: <ACPI PCI bus> numa-domain 0 on pcib8
igb0: <Intel(R) I350 (Copper)> mem 0x91e00000-0x91efffff,0x91f0c000-0x91f0ffff at device 0.0 numa-domain 0 on pci8
igb0: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000faa
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 8 RX queues 8 TX queues
igb0: Using MSI-X interrupts with 9 vectors
igb0: Ethernet address: ec:f4:bb:ef:c9:54
igb0: netmap queues/slots: TX 8/1024, RX 8/1024
igb1: <Intel(R) I350 (Copper)> mem 0x91d00000-0x91dfffff,0x91f08000-0x91f0bfff at device 0.1 numa-domain 0 on pci8
igb1: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000faa
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 8 RX queues 8 TX queues
igb1: Using MSI-X interrupts with 9 vectors
igb1: Ethernet address: ec:f4:bb:ef:c9:55
igb1: netmap queues/slots: TX 8/1024, RX 8/1024
igb2: <Intel(R) I350 (Copper)> mem 0x91c00000-0x91cfffff,0x91f04000-0x91f07fff at device 0.2 numa-domain 0 on pci8
igb2: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000faa
igb2: Using 1024 TX descriptors and 1024 RX descriptors
igb2: Using 8 RX queues 8 TX queues
igb2: Using MSI-X interrupts with 9 vectors
igb2: Ethernet address: ec:f4:bb:ef:c9:56
igb2: netmap queues/slots: TX 8/1024, RX 8/1024
igb3: <Intel(R) I350 (Copper)> mem 0x91b00000-0x91bfffff,0x91f00000-0x91f03fff at device 0.3 numa-domain 0 on pci8
igb3: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000faa
igb3: Using 1024 TX descriptors and 1024 RX descriptors
igb3: Using 8 RX queues 8 TX queues
igb3: Using MSI-X interrupts with 9 vectors
igb3: Ethernet address: ec:f4:bb:ef:c9:57
igb3: netmap queues/slots: TX 8/1024, RX 8/1024
pcib9: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 0 on pci2
pci9: <ACPI PCI bus> numa-domain 0 on pcib9
nvme4: <Generic NVMe Device> mem 0x91a00000-0x91a03fff,0x91a04000-0x91a040ff at device 0.0 numa-domain 0 on pci9
pci2: <unknown> at device 17.0 (no driver attached)
ahci0: <Intel Wellsburg AHCI SATA controller> port 0x3078-0x307f,0x308c-0x308f,0x3070-0x3077,0x3088-0x308b,0x3040-0x305f mem 0x92601000-0x926017ff at device 17.4 numa-domain 0 on pci2
ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
pci2: <simple comms> at device 22.0 (no driver attached)
pci2: <simple comms> at device 22.1 (no driver attached)
ehci0: <Intel Wellsburg USB 2.0 controller> mem 0x92603000-0x926033ff at device 26.0 numa-domain 0 on pci2
usbus0: EHCI version 1.0
usbus0 numa-domain 0 on ehci0
usbus0: 480Mbps High Speed USB v2.0
pcib10: <PCI-PCI bridge> at device 28.0 numa-domain 0 on pci2
pci10: <PCI bus> numa-domain 0 on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 28.7 numa-domain 0 on pci2
pci11: <ACPI PCI bus> numa-domain 0 on pcib11
pcib12: <PCI-PCI bridge> at device 0.0 numa-domain 0 on pci11
pci12: <PCI bus> numa-domain 0 on pcib12
pcib13: <PCI-PCI bridge> at device 0.0 numa-domain 0 on pci12
pci13: <PCI bus> numa-domain 0 on pcib13
pcib14: <PCI-PCI bridge> at device 0.0 numa-domain 0 on pci13
pci14: <PCI bus> numa-domain 0 on pcib14
vgapci0: <VGA-compatible display> mem 0x90000000-0x90ffffff,0x91800000-0x91803fff,0x91000000-0x917fffff at device 0.0 numa-domain 0 on pci14
vgapci0: Boot video device
ehci1: <Intel Wellsburg USB 2.0 controller> mem 0x92602000-0x926023ff at device 29.0 numa-domain 0 on pci2
usbus1: EHCI version 1.0
usbus1 numa-domain 0 on ehci1
usbus1: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci2
isa0: <ISA bus> numa-domain 0 on isab0
ahci1: <Intel Wellsburg AHCI SATA controller> port 0x3068-0x306f,0x3084-0x3087,0x3060-0x3067,0x3080-0x3083,0x3020-0x303f mem 0x92600000-0x926007ff at device 31.2 numa-domain 0 on pci2
ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich4: <AHCI channel> at channel 0 on ahci1
ahcich5: <AHCI channel> at channel 1 on ahci1
ahcich6: <AHCI channel> at channel 2 on ahci1
ahcich7: <AHCI channel> at channel 3 on ahci1
ahcich8: <AHCI channel> at channel 4 on ahci1
ahcich9: <AHCI channel> at channel 5 on ahci1
ahciem1: <AHCI enclosure management bridge> on ahci1
pcib15: <ACPI Host-PCI bridge> numa-domain 1 on acpi0
pci15: <ACPI PCI bus> numa-domain 1 on pcib15
pcib16: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 1 on pci15
pci16: <ACPI PCI bus> numa-domain 1 on pcib16
ix0: <Intel(R) X520 82599 (Dual SFP+)> port 0x9020-0x903f mem 0xc8600000-0xc86fffff,0xc8704000-0xc8707fff at device 0.0 numa-domain 1 on pci16
WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics. Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter. Intel Corporation is not responsible for any harm caused by using untested modules.
ix0: Using 2048 TX descriptors and 2048 RX descriptors
ix0: Using 16 RX queues 16 TX queues
ix0: Using MSI-X interrupts with 17 vectors
ix0: allocated for 16 queues
ix0: allocated for 16 rx queues
ix0: Ethernet address: a0:36:9f:83:e1:14
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: Option ROM V19-b5-p12 eTrack 0x8000095d
ix0: netmap queues/slots: TX 16/2048, RX 16/2048
ix1: <Intel(R) X520 82599 (Dual SFP+)> port 0x9000-0x901f mem 0xc8500000-0xc85fffff,0xc8700000-0xc8703fff at device 0.1 numa-domain 1 on pci16
ix1: Using 2048 TX descriptors and 2048 RX descriptors
ix1: Using 16 RX queues 16 TX queues
ix1: Using MSI-X interrupts with 17 vectors
ix1: allocated for 16 queues
ix1: allocated for 16 rx queues
ix1: Ethernet address: a0:36:9f:83:e1:16
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: Option ROM V19-b5-p12 eTrack 0x8000095d
ix1: netmap queues/slots: TX 16/2048, RX 16/2048
pcib17: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 1 on pci15
pci17: <ACPI PCI bus> numa-domain 1 on pcib17
ahci2: <ASMedia ASM1062 AHCI SATA controller> port 0x8028-0x802f,0x8034-0x8037,0x8020-0x8027,0x8030-0x8033,0x8000-0x801f mem 0xc8400000-0xc84001ff at device 0.0 numa-domain 1 on pci17
ahci2: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported
ahci2: quirks=0xc00000<NOCCS,NOAUX>
ahcich10: <AHCI channel> at channel 0 on ahci2
ahcich11: <AHCI channel> at channel 1 on ahci2
pcib18: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 1 on pci15
pci18: <ACPI PCI bus> numa-domain 1 on pcib18
nvme5: <Generic NVMe Device> mem 0xc8300000-0xc8303fff at device 0.0 numa-domain 1 on pci18
pcib19: <ACPI PCI-PCI bridge> at device 3.1 numa-domain 1 on pci15
pci19: <ACPI PCI bus> numa-domain 1 on pcib19
nvme6: <Generic NVMe Device> mem 0xc8200000-0xc8203fff at device 0.0 numa-domain 1 on pci19
pcib20: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 1 on pci15
pci20: <ACPI PCI bus> numa-domain 1 on pcib20
nvme7: <Generic NVMe Device> mem 0xc8100000-0xc8103fff at device 0.0 numa-domain 1 on pci20
pcib21: <ACPI PCI-PCI bridge> at device 3.3 numa-domain 1 on pci15
pci21: <ACPI PCI bus> numa-domain 1 on pcib21
nvme8: <Generic NVMe Device> mem 0xc8000000-0xc8003fff at device 0.0 numa-domain 1 on pci21
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROM> at iomem 0xed800-0xf17ff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
cpufreq0: <CPU frequency control> on cpu0
cpufreq1: <CPU frequency control> on cpu1
cpufreq2: <CPU frequency control> on cpu2
cpufreq3: <CPU frequency control> on cpu3
cpufreq4: <CPU frequency control> on cpu4
cpufreq5: <CPU frequency control> on cpu5
cpufreq6: <CPU frequency control> on cpu6
cpufreq7: <CPU frequency control> on cpu7
cpufreq8: <CPU frequency control> on cpu8
cpufreq9: <CPU frequency control> on cpu9
cpufreq10: <CPU frequency control> on cpu10
cpufreq11: <CPU frequency control> on cpu11
cpufreq12: <CPU frequency control> on cpu12
cpufreq13: <CPU frequency control> on cpu13
cpufreq14: <CPU frequency control> on cpu14
cpufreq15: <CPU frequency control> on cpu15
cpufreq16: <CPU frequency control> on cpu16
cpufreq17: <CPU frequency control> on cpu17
cpufreq18: <CPU frequency control> on cpu18
cpufreq19: <CPU frequency control> on cpu19
cpufreq20: <CPU frequency control> on cpu20
cpufreq21: <CPU frequency control> on cpu21
cpufreq22: <CPU frequency control> on cpu22
cpufreq23: <CPU frequency control> on cpu23
cpufreq24: <CPU frequency control> on cpu24
cpufreq25: <CPU frequency control> on cpu25
cpufreq26: <CPU frequency control> on cpu26
cpufreq27: <CPU frequency control> on cpu27
cpufreq28: <CPU frequency control> on cpu28
cpufreq29: <CPU frequency control> on cpu29
cpufreq30: <CPU frequency control> on cpu30
cpufreq31: <CPU frequency control> on cpu31
cpufreq32: <CPU frequency control> on cpu32
cpufreq33: <CPU frequency control> on cpu33
cpufreq34: <CPU frequency control> on cpu34
cpufreq35: <CPU frequency control> on cpu35
cpufreq36: <CPU frequency control> on cpu36
cpufreq37: <CPU frequency control> on cpu37
cpufreq38: <CPU frequency control> on cpu38
cpufreq39: <CPU frequency control> on cpu39
cpufreq40: <CPU frequency control> on cpu40
cpufreq41: <CPU frequency control> on cpu41
cpufreq42: <CPU frequency control> on cpu42
cpufreq43: <CPU frequency control> on cpu43
cpufreq44: <CPU frequency control> on cpu44
cpufreq45: <CPU frequency control> on cpu45
cpufreq46: <CPU frequency control> on cpu46
cpufreq47: <CPU frequency control> on cpu47
cpufreq48: <CPU frequency control> on cpu48
cpufreq49: <CPU frequency control> on cpu49
cpufreq50: <CPU frequency control> on cpu50
cpufreq51: <CPU frequency control> on cpu51
cpufreq52: <CPU frequency control> on cpu52
cpufreq53: <CPU frequency control> on cpu53
cpufreq54: <CPU frequency control> on cpu54
cpufreq55: <CPU frequency control> on cpu55
cpufreq56: <CPU frequency control> on cpu56
cpufreq57: <CPU frequency control> on cpu57
cpufreq58: <CPU frequency control> on cpu58
cpufreq59: <CPU frequency control> on cpu59
cpufreq60: <CPU frequency control> on cpu60
cpufreq61: <CPU frequency control> on cpu61
cpufreq62: <CPU frequency control> on cpu62
cpufreq63: <CPU frequency control> on cpu63
cpufreq64: <CPU frequency control> on cpu64
cpufreq65: <CPU frequency control> on cpu65
cpufreq66: <CPU frequency control> on cpu66
cpufreq67: <CPU frequency control> on cpu67
cpufreq68: <CPU frequency control> on cpu68
cpufreq69: <CPU frequency control> on cpu69
cpufreq70: <CPU frequency control> on cpu70
cpufreq71: <CPU frequency control> on cpu71
Timecounter "TSC-low" frequency 1149997214 Hz quality 1000
Timecounters tick every 1.000 msec
ugen0.1: <Intel EHCI root HUB> at usbus0
uhub0 numa-domain 0 on usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel EHCI root HUB> at usbus1
uhub1 numa-domain 0 on usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
mrsas0: Disestablish mrsas intr hook
nvme4: Allocated 200MB host memory buffer
nvme5: Allocated 64MB host memory buffer
nvme8: Allocated 64MB host memory buffer
nda0 at nvme0 bus 0 scbus16 target 0 lun 1
nda0: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y722875X>
nda0: Serial Number S7KGNU0Y722875X
nda0: nvme version 2.0
nda0: 3815447MB (7814037168 512 byte sectors)
nda1 at nvme1 bus 0 scbus17 target 0 lun 1
nda1: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y915666E>
nda1: Serial Number S7KGNU0Y915666E
nda1: nvme version 2.0
nda1: 3815447MB (7814037168 512 byte sectors)
nda2 at nvme2 bus 0 scbus18 target 0 lun 1
nda2: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y912937J>
nda2: Serial Number S7KGNU0Y912937J
nda2: nvme version 2.0
nda2: 3815447MB (7814037168 512 byte sectors)
nda3 at nvme3 bus 0 scbus19 target 0 lun 1
nda3: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y912955D>
nda3: Serial Number S7KGNU0Y912955D
nda3: nvme version 2.0
nda3: 3815447MB (7814037168 512 byte sectors)
nda4 at nvme4 bus 0 scbus20 target 0 lun 1
nda4: <WDC WDS250G2B0C-00PXH0 233010WD 21280E800995>
nda4: Serial Number 21280E800995
nda4: nvme version 1.4
nda4: 238475MB (488397168 512 byte sectors)
nda5 at nvme5 bus 0 scbus21 target 0 lun 1
nda5: <Samsung SSD 990 EVO Plus 4TB 2B2QKXG7 S7U8NJ0Y716854P>
nda5: Serial Number S7U8NJ0Y716854P
nda5: nvme version 2.0
nda5: 3815447MB (7814037168 512 byte sectors)
nda6 at nvme6 bus 0 scbus22 target 0 lun 1
nda6: <Samsung SSD 980 PRO with Heatsink 1TB 4B2QGXA7 S6WSNJ0T208743F>
nda6: Serial Number S6WSNJ0T208743F
nda6: nvme version 1.3
nda6: 953869MB (1953525168 512 byte sectors)
nda7 at nvme7 bus 0 scbus23 target 0 lun 1
nda7: <Samsung SSD 980 PRO with Heatsink 1TB 4B2QGXA7 S6WSNJ0T207774T>
nda7: Serial Number S6WSNJ0T207774T
nda7: nvme version 1.3
nda7: 953869MB (1953525168 512 byte sectors)
nda8 at nvme8 bus 0 scbus24 target 0 lun 1
nda8: <Samsung SSD 990 EVO Plus 4TB 2B2QKXG7 S7U8NJ0Y716801F>
nda8: Serial Number S7U8NJ0Y716801F
nda8: nvme version 2.0
nda8: 3815447MB (7814037168 512 byte sectors)
ada0 at ahcich8 bus 0 scbus11 target 0 lun 0
ada0: <SATADOM-ML 3MG2-P M150821i> ACS-2 ATA SATA 3.x device
ada0: Serial Number 20170718AA0000185556
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
ada0: Command Queueing enabled
ada0: 118288MB (242255664 512 byte sectors)
ada1 at ahcich9 bus 0 scbus12 target 0 lun 0
ada1: <SATADOM-ML 3MG2-P M150821i> ACS-2 ATA SATA 3.x device
ada1: Serial Number 20170719AA1178164201
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
ada1: Command Queueing enabled
ada1: 118288MB (242255664 512 byte sectors)
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ses1 at ahciem1 bus 0 scbus13 target 0 lun 0
ses1: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses1: SEMB SES Device
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da0: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number Y7P0A02LTEVE
da0: 150.000MB/s transfers
da0: 763097MB (1562824368 512 byte sectors)
da2 at mrsas0 bus 1 scbus1 target 2 lun 0
da2: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number Y7P0A033TEVE
da2: 150.000MB/s transfers
da2: 763097MB (1562824368 512 byte sectors)
da1 at mrsas0 bus 1 scbus1 target 1 lun 0
da1: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number Y7P0A02MTEVE
da1: 150.000MB/s transfers
da1: 763097MB (1562824368 512 byte sectors)
da3 at mrsas0 bus 1 scbus1 target 3 lun 0
da3: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number Y7P0A022TEVE
da3: 150.000MB/s transfers
da3: 763097MB (1562824368 512 byte sectors)
ses1: ada0,pass15 in 'Slot 04', SATA Slot: scbus11 target 0
ses1: ada1,pass16 in 'Slot 05', SATA Slot: scbus12 target 0
da5 at mrsas0 bus 1 scbus1 target 5 lun 0
da5: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number Y7P0A02ATEVE
da5: 150.000MB/s transfers
da5: 763097MB (1562824368 512 byte sectors)
da4 at mrsas0 bus 1 scbus1 target 4 lun 0
da4: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number Y7P0A02QTEVE
da4: 150.000MB/s transfers
da4: 763097MB (1562824368 512 byte sectors)
da7 at mrsas0 bus 1 scbus1 target 7 lun 0
da7: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number Y7P0A02DTEVE
da7: 150.000MB/s transfers
da7: 763097MB (1562824368 512 byte sectors)
da6 at mrsas0 bus 1 scbus1 target 6 lun 0
da6: <TOSHIBA PX05SMB080Y AS0B> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number Y7P0A02GTEVE
da6: 150.000MB/s transfers
da6: 763097MB (1562824368 512 byte sectors)
da12 at mrsas0 bus 1 scbus1 target 12 lun 0
da12: <ATA Samsung SSD 870 3B6Q> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number S757NS0Y700758M
da12: 150.000MB/s transfers
da12: 3815447MB (7814037168 512 byte sectors)
da12: quirks=0x8<4K>
da13 at mrsas0 bus 1 scbus1 target 13 lun 0
da13: <ATA Samsung SSD 870 3B6Q> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number S757NS0Y700760R
da13: 150.000MB/s transfers
da13: 3815447MB (7814037168 512 byte sectors)
da13: quirks=0x8<4K>
da11 at mrsas0 bus 1 scbus1 target 11 lun 0
da11: <ATA WDC  WDS400T2B0A 20WD> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number 230151800473
da11: 150.000MB/s transfers
da11: 3815447MB (7814037168 512 byte sectors)
da8 at mrsas0 bus 1 scbus1 target 8 lun 0
da8: <ATA WDC  WDS400T2B0A 20WD> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number 230151801478
da8: 150.000MB/s transfers
da8: 3815447MB (7814037168 512 byte sectors)
da10 at mrsas0 bus 1 scbus1 target 10 lun 0
da10: <ATA WDC  WDS400T2B0A 20WD> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number 22492H800867
da10: 150.000MB/s transfers
da10: 3815447MB (7814037168 512 byte sectors)
da9 at mrsas0 bus 1 scbus1 target 9 lun 0
da9: <ATA WDC  WDS400T2B0A 20WD> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number 230151801284
da9: 150.000MB/s transfers
da9: 3815447MB (7814037168 512 byte sectors)
GEOM_MIRROR: Device mirror/swap launched (2/2).
Trying to mount root from zfs:zroot/ROOT/default []...
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
ugen1.2: <vendor 0x8087 product 0x8002> at usbus1
uhub2 numa-domain 0 on uhub1
uhub2: <vendor 0x8087 product 0x8002, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen0.2: <vendor 0x8087 product 0x800a> at usbus0
uhub3 numa-domain 0 on uhub0
uhub3: <vendor 0x8087 product 0x800a, class 9/0, rev 2.00/0.05, addr 2> on usbus0
Root mount waiting for: usbus0 usbus1
uhub3: 6 ports with 6 removable, self powered
uhub2: 8 ports with 8 removable, self powered
ugen0.3: <no manufacturer Gadget USB HUB> at usbus0
uhub4 numa-domain 0 on uhub3
uhub4: <no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, addr 3> on usbus0
ugen1.3: <ATEN UC-10KM V1.3.124> at usbus1
ukbd0 numa-domain 0 on uhub2
ukbd0: <ATEN UC-10KM V1.3.124, class 0/0, rev 1.10/1.00, addr 3> on usbus1
kbd1 at ukbd0
uhub4: 6 ports with 6 removable, self powered
Root mount waiting for: usbus0
ugen0.4: <Avocent Keyboard/Mouse Function> at usbus0
ukbd1 numa-domain 0 on uhub4
ukbd1: <Avocent Keyboard/Mouse Function, class 0/0, rev 2.00/0.00, addr 4> on usbus0
kbd2 at ukbd1
GEOM_ELI: Device mirror/swap.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: accelerated software
acpi_wmi0: <ACPI-WMI mapping> on acpi0
acpi_wmi0: cannot find EC device
acpi_wmi1: <ACPI-WMI mapping> on acpi0
acpi_wmi1: cannot find EC device
CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz (2299.99-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306f2  Family=0x6  Model=0x3f  Stepping=2
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x37ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,NFPUSG>
  Structured Extended Features3=0x9c000400<MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
bridge0: Ethernet address: 58:9c:fc:10:8c:57
bridge0: link state changed to UP
ix0: promiscuous mode enabled
lo0: link state changed to UP
lo1: link state changed to UP
lo2: link state changed to UP
ix0: link state changed to UP
pflog0: promiscuous mode enabled
ums0 numa-domain 0 on uhub4
ums0: <Avocent Keyboard/Mouse Function, class 0/0, rev 2.00/0.00, addr 4> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
ums1 numa-domain 0 on uhub4
ums1: <Avocent Keyboard/Mouse Function, class 0/0, rev 2.00/0.00, addr 4> on usbus0
ums1: 3 buttons and [XYZ] coordinates ID=0
ums2 numa-domain 0 on uhub2
ums2: <ATEN UC-10KM V1.3.124, class 0/0, rev 1.10/1.00, addr 3> on usbus1
ums2: 5 buttons and [XYZ] coordinates ID=0
tap0: Ethernet address: 58:9c:fc:00:77:5a
Security policy loaded: MAC/ntpd (mac_ntpd)
tap0: promiscuous mode enabled
ix0: link state changed to DOWN
bridge0: 2 link states coalesced
bridge0: link state changed to UP
tap0: link state changed to UP
ix0: link state changed to UP
epair29a: Ethernet address: 02:d2:fe:70:bc:0a
epair29b: Ethernet address: 02:d2:fe:70:bc:0b
epair29a: link state changed to UP
epair29b: link state changed to UP
epair29a: promiscuous mode enabled
ix0: link state changed to DOWN
ix0: link state changed to UP
lo0: link state changed to UP
lo3: link state changed to UP
ZFS WARNING: Unable to attach to nda6.
ZFS WARNING: Unable to attach to nda7.
ZFS WARNING: Unable to attach to nda7.
ZFS WARNING: Unable to attach to nda6.
ugen0.4: <Avocent Keyboard/Mouse Function> at usbus0 (disconnected)
ukbd1: at uhub4, port 1, addr 4 (disconnected)
ukbd1: detached
ums0: at uhub4, port 1, addr 4 (disconnected)
ums0: detached
ums1: at uhub4, port 1, addr 4 (disconnected)
ums1: detached
ZFS WARNING: Unable to attach to nda7.
ZFS WARNING: Unable to attach to nda6.

sesutil show

[16:57 r730-01 dvl ~] % sudo sesutil show
ses0: ; ID: 3061686369656d30
Desc            Dev     Model                     Ident                Size/Status
Slot 00         -       -                         -                    Not Installed
Slot 01         -       -                         -                    Not Installed
Slot 02         -       -                         -                    Not Installed
Slot 03         -       -                         -                    Not Installed

ses1: ; ID: 3061686369656d31
Desc            Dev     Model                     Ident                Size/Status
Slot 00         -       -                         -                    Not Installed
Slot 01         -       -                         -                    Not Installed
Slot 02         -       -                         -                    Not Installed
Slot 03         -       -                         -                    Not Installed
Slot 04         ada0    SATADOM-ML 3MG2-P         20170718AA0000185556 124G
Slot 05         ada1    SATADOM-ML 3MG2-P         20170719AA1178164201 124G

jls

[16:57 r730-01 dvl ~] % jls
   JID  IP Address      Hostname                      Path
     1  10.55.0.73      dns1.int.unixathome.org       /jails/dns1
     2  10.55.0.44      cliff2.int.unixathome.org     /jails/cliff2
     3  10.55.0.151     mysql01.int.unixathome.org    /jails/mysql01
     4  10.55.0.150     pg01.int.unixathome.org       /jails/pg01
     5  10.55.0.32      pg02.int.unixathome.org       /jails/pg02
     6  10.55.0.34      pg03.int.unixathome.org       /jails/pg03
     7                  pkg01.int.unixathome.org      /jails/pkg01
     8  10.55.0.37      dev-ingress01.int.unixathome. /jails/dev-ingress01
     9  10.55.0.37      freshports                    /jails/dev-ingress01/jails/freshports
    10  10.55.0.39      dev-nginx01.int.unixathome.or /jails/dev-nginx01
    11  10.55.0.81      dvl-ingress01.int.unixathome. /jails/dvl-ingress01
    12  10.55.0.81      freshports                    /jails/dvl-ingress01/jails/freshports
    13  10.55.0.82      dvl-nginx01.int.unixathome.or /jails/dvl-nginx01
    14  10.55.0.40      test-ingress01.int.unixathome /jails/test-ingress01
    15  10.55.0.40      freshports                    /jails/test-ingress01/jails/freshports
    16  10.55.0.42      test-nginx01.int.unixathome.o /jails/test-nginx01
    17  10.55.0.45      stage-ingress01.int.unixathom /jails/stage-ingress01
    18  10.55.0.45      freshports                    /jails/stage-ingress01/jails/freshports
    19  10.55.0.46      stage-nginx01.int.unixathome. /jails/stage-nginx01
    20  10.55.0.4       bacula.int.unixathome.org     /jails/bacula
    21  10.55.0.27      besser.int.unixathome.org     /jails/besser
    22  10.55.0.54      certs-rsync.int.unixathome.or /jails/certs-rsync
    23  10.55.0.112     certs.int.unixathome.org      /jails/certs
    24  10.55.0.30      git.langille.org              /jails/git
    25  10.55.0.6       svn.int.unixathome.org        /jails/svn
    26  10.55.0.3       webserver.int.unixathome.org  /jails/webserver
    27  10.55.0.10      mqtt01.int.unixathome.org     /jails/mqtt01
    28  10.55.0.33      bacula-sd-02.int.unixathome.o /jails/bacula-sd-02
    29  10.55.0.28      talos.int.unixathome.org      /jails/talos
    30  10.55.0.50      samdrucker.int.unixathome.org /jails/samdrucker
    32  10.55.0.16      mydev.int.unixathome.org      /jails/mydev
    33  10.55.0.31      jail_within_jail.int.unixatho /jails/jail_within_jail
    34  10.55.0.24      serpico.int.unixathome.org    /jails/serpico
    35  10.55.0.53      dns-hidden-master.int.unixath /jails/dns-hidden-master
    36  10.55.0.20      nsnotify.int.unixathome.org   /jails/nsnotify
    37  10.55.0.131     unifi01.int.unixathome.org    /jails/unifi01
    38  10.55.0.49      bacula-sd-03.int.unixathome.o /jails/bacula-sd-03
Top

FreeBSD Foundation Q3 2025 Status Update

Post by FreeBSD Foundation via FreeBSD Foundation »

Written as part of the FreeBSD Project’s 3rd Quarter 2025 Status Report, check out the highlights of what we did to help FreeBSD last quarter:

The FreeBSD Foundation is a 501(c)(3) non-profit dedicated to advancing FreeBSD through both technical and non-technical support. Funded entirely by donations, the Foundation supports software development, infrastructure, security, and collaboration efforts; organizes events and developer summits; provides educational resources; and represents the FreeBSD Project in legal matters.

Here are some of the ways we supported FreeBSD in the third quarter of 2025.

OS Improvements

The Foundation continued to support two major initiatives: the Laptop Support and Usability project (in collaboration with Quantum Leap Research) and an infrastructure modernization project commissioned by the Sovereign Tech Agency. For background on both efforts, see the 2025Q1 quarterly status report.

Throughout the quarter, there were 451 src, 71 ports, and 25 doc commits that identified the FreeBSD Foundation as a sponsor.

Refer to the following report entries describing much of that committed development work:

Other highlights include:

  • Improved virtual memory scalability, allowing multiple processes to load shared libraries in parallel.
  • Greater UFS reliability on very large filesystems (with more than 2 billion inodes).
  • Support for systems with over 4 TB of RAM, including a reworked Kernel Virtual Address (KVA) layout tailored for the 57-bit address space (LA57) architecture.
  • Kqueue inheritance across fork(2).
  • A new safeguard (noshutdown) to help prevent accidental system shutdown.
  • Simplified and more reliable filesystem rename operations.
  • A fix for amd64 pmap panics under low-memory conditions.
  • Numerous fixes for race conditions in timeout(1).
  • A new EXTERROR(9) interface, standardizing how external applications can report detailed error information.

FreeBSD, under the management of the Foundation, participated in its 21st consecutive Google Summer of Code (GSoC) program. All twelve projects were successfully completed.

Four of those GSoC participants contributed report entries:

Continuous Integration and Workflow Improvement

As part of our continued support of the FreeBSD Project, the Foundation supports a full-time staff member dedicated to improving the Project’s continuous integration system and test infrastructure.

Advocacy

Advocacy work in the 3rd quarter of 2025 included representing FreeBSD at open source events, producing more technical tutorials and working on the upcoming November 2025 FreeBSD Vendor Summit. Take a look at just a few of the ways the Foundation helped advocate for FreeBSD in Q3 of 2025:

  • Sponsored and attended EuroBSDcon 2025, held in Zagreb, Croatia; September 25-28, 2025.
  • Members of the Foundation team presented at the Open Source Summit, Europe, August 25-27, 2025, Amsterdam, Netherlands.
  • Planning continued for the November 2025 FreeBSD Vendor Summit, taking place November 6-7, 2025 in San Jose, CA. Registration is open and the schedule is available.
  • Published the following blogs and videos to help to inform and educate the community:

Fundraising

Thank you to everyone who gave us a financial contribution last quarter to help fund our work to support the Project. We are grateful for your investment in FreeBSD!

Please consider supporting our efforts in 2025 by making a donation here: https://freebsdfoundation.org/donate/.

Or, check out our Partnership opportunities here: https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/.

Legal/FreeBSD IP

The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

Go to https://freebsdfoundation.org to find more about how we support FreeBSD and how we can help you!

The post FreeBSD Foundation Q3 2025 Status Update first appeared on FreeBSD Foundation.

Top

data04: moving data01/bacula/volumes over

Post by Dan Langille via Dan Langille's Other Diary »

After finally creating data04, it’s time to relocate some data.

In this post:

  • FreeBSD 14.3

The source

The current zpool which is straining is data01:

[20:04 r730-01 dvl ~] % zfs list -r data01
NAME                                       USED  AVAIL  REFER  MOUNTPOINT
data01                                    3.91T   102G   205K  none
data01/bacula                             3.62T   102G   188K  /jails/bacula-sd-03/usr/local/bacula
data01/bacula/volumes                     3.62T   102G   205K  /jails/bacula-sd-03/usr/local/bacula/volumes
data01/bacula/volumes/DiffFile-03          549G   102G   549G  /jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03
data01/bacula/volumes/FullFile-03         2.61T   102G  2.60T  /jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03
data01/bacula/volumes/IncrFile-03          492G   102G   492G  /jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03
data01/bacula/working                      247K   102G   247K  /jails/bacula-sd-03/usr/local/bacula/working
data01/jail_within_jail                    617M   102G   205K  none
data01/jail_within_jail/jails              617M   102G   205K  none
data01/jail_within_jail/jails/freshports   617M   102G   617M  none
data01/mkjail                             1.83G   102G   468M  /var/db/mkjail
data01/mkjail/14.3-RELEASE                1.37G   102G  1.37G  /var/db/mkjail/14.3-RELEASE
data01/reserved                            290G   392G   205K  none

I plan to move data01/bacula/volumes over first, then data01/bacula/working.

Let’s modify the source for better results on the move.

What do we have now?

[20:22 r730-01 dvl /jails/bacula-sd-03/usr/local/bacula/volumes] % zfs get compression,recordsize data01/bacula/volumes
NAME                   PROPERTY     VALUE           SOURCE
data01/bacula/volumes  compression  on              inherited from data01
data01/bacula/volumes  recordsize   1M              local

Let’s make that zstd and something bigger. How much bigger? I don’t know. Let’s try:

[20:19 r730-01 dvl ~] % sudo zfs set recordsize=2M data04
[20:19 r730-01 dvl ~] % sudo zfs set recordsize=5M data04
cannot set property for 'data04': 'recordsize' must be power of 2 from 512B to 16M

This is all backup data. Files are generally > 16M, much larger, more like 20G. A larger record size make sense here.

[20:24 r730-01 dvl /jails/bacula-sd-03/usr/local/bacula/volumes] % sudo zfs set recordsize=16M compression=zstd data01/bacula/volumes/DiffFile-03
[20:24 r730-01 dvl /jails/bacula-sd-03/usr/local/bacula/volumes] % sudo zfs set recordsize=16M compression=zstd data01/bacula/volumes/FullFile-03
[20:24 r730-01 dvl /jails/bacula-sd-03/usr/local/bacula/volumes] % sudo zfs set recordsize=16M compression=zstd data01/bacula/volumes/IncrFile-03

Those change do not affect data at rest, only future writes. Like the send | recv I’m about to do.

Sending the data

When sending the data, I want to be sure the data was not modified. I stopped the jail which used it:

[20:24 r730-01 dvl /jails/bacula-sd-03/usr/local/bacula/volumes] % sudo service jail stop bacula-sd-03 
Stopping jails: bacula-sd-03.

The snapshot:

root@r730-01:/jails/bacula-sd-03/usr/local/bacula/volumes # zfs snapshot -r data01/bacula/volumes@for-copy-to-data04
root@r730-01:/jails/bacula-sd-03/usr/local/bacula/volumes # zfs send -Rv data01/bacula/volumes@for-copy-to-data04 | zfs receive -uFv data04
full send of data01/bacula/volumes@for-copy-to-data04 estimated size is 39.6K
full send of data01/bacula/volumes/FullFile-03@autosnap_2025-11-16_00:00:16_daily estimated size is 3.74T
send from @autosnap_2025-11-16_00:00:16_daily to data01/bacula/volumes/FullFile-03@autosnap_2025-11-17_00:00:01_daily estimated size is 624B
send from @autosnap_2025-11-17_00:00:01_daily to data01/bacula/volumes/FullFile-03@autosnap_2025-11-18_00:00:02_daily estimated size is 624B
send from @autosnap_2025-11-18_00:00:02_daily to data01/bacula/volumes/FullFile-03@autosnap_2025-11-19_00:00:13_daily estimated size is 624B
send from @autosnap_2025-11-19_00:00:13_daily to data01/bacula/volumes/FullFile-03@autosnap_2025-11-20_00:00:03_daily estimated size is 624B
...
received 312B stream in 0.11 seconds (2.82K/sec)
receiving incremental stream of data01/bacula/volumes/DiffFile-03@autosnap_2025-12-09_00:00:12_daily into data04/DiffFile-03@autosnap_2025-12-09_00:00:12_daily
received 312B stream in 0.12 seconds (2.44K/sec)
receiving incremental stream of data01/bacula/volumes/DiffFile-03@for-copy-to-data04 into data04/DiffFile-03@for-copy-to-data04
received 312B stream in 0.09 seconds (3.37K/sec)
root@r730-01:/jails/bacula-sd-03/usr/local/bacula/volumes # 

What did we get?

Here’s what we have:

root@r730-01:~ # zfs list -r data04
NAME                 USED  AVAIL  REFER  MOUNTPOINT
data04              3.19T  17.4T   205K  /data04
data04/DiffFile-03   405G  17.4T   405G  /data04/DiffFile-03
data04/FullFile-03  2.45T  17.4T  2.44T  /data04/FullFile-03
data04/IncrFile-03   360G  17.4T   360G  /data04/IncrFile-03

Oh, no volumes name in there. That will have to change. Compare with:

[23:09 r730-01 dvl ~] % zfs list -r data01/bacula/volumes
NAME                                USED  AVAIL  REFER  MOUNTPOINT
data01/bacula/volumes              3.62T   102G   205K  /jails/bacula-sd-03/usr/local/bacula/volumes
data01/bacula/volumes/DiffFile-03   549G   102G   549G  /jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03
data01/bacula/volumes/FullFile-03  2.61T   102G  2.60T  /jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03
data01/bacula/volumes/IncrFile-03   492G   102G   492G  /jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03

Those are impressive changes. (remember it’s after and before above us there…)

  1. DiffFile-03 – from 549G to 405G (savings of 26%)
  2. FullFile-03 – From 2.61T to 2.45T (savings of 6%)
  3. IncrFile-03 – From 492G to 360G (savings of 26%)

Adjusting things within data04

Let’s get back the old structure we had:

root@r730-01:~ # zfs create data04/bacula
root@r730-01:~ # zfs create data04/bacula/volumes
root@r730-01:~ # zfs rename data04/DiffFile-03 data04/bacula/volumes/DiffFile-03
root@r730-01:~ # zfs rename data04/FullFile-03 data04/bacula/volumes/FullFile-03
root@r730-01:~ # zfs rename data04/IncrFile-03 data04/bacula/volumes/IncrFile-03
root@r730-01:~ # zfs list -r data04
NAME                                USED  AVAIL  REFER  MOUNTPOINT
data04                             3.19T  17.4T   205K  /data04
data04/bacula                      3.19T  17.4T   205K  /data04/bacula
data04/bacula/volumes              3.19T  17.4T   205K  /data04/bacula/volumes
data04/bacula/volumes/DiffFile-03   405G  17.4T   405G  /data04/bacula/volumes/DiffFile-03
data04/bacula/volumes/FullFile-03  2.45T  17.4T  2.44T  /data04/bacula/volumes/FullFile-03
data04/bacula/volumes/IncrFile-03   360G  17.4T   360G  /data04/bacula/volumes/IncrFile-03
root@r730-01:~ # 

Next, let’s look at inheritance.

[23:26 r730-01 dvl ~] % zfs get -t filesystem -r recordsize data04       
NAME                               PROPERTY    VALUE    SOURCE
data04                             recordsize  16M      received
data04/bacula                      recordsize  16M      inherited from data04
data04/bacula/volumes              recordsize  16M      local
data04/bacula/volumes/DiffFile-03  recordsize  16M      received
data04/bacula/volumes/FullFile-03  recordsize  16M      received
data04/bacula/volumes/IncrFile-03  recordsize  16M      received

Let’s make volumes the source of truth.

[23:26 r730-01 dvl ~] % sudo zfs inherit recordsize data04/bacula/volumes/DiffFile-03
[23:26 r730-01 dvl ~] % sudo zfs inherit recordsize data04/bacula/volumes/FullFile-03
[23:26 r730-01 dvl ~] % sudo zfs inherit recordsize data04/bacula/volumes/IncrFile-  
cannot open 'data04/bacula/volumes/IncrFile-': dataset does not exist
[23:26 r730-01 dvl ~] % sudo zfs inherit recordsize data04/bacula/volumes/IncrFile-03
[23:27 r730-01 dvl ~] % zfs get -t filesystem -r recordsize data04                   
NAME                               PROPERTY    VALUE    SOURCE
data04                             recordsize  16M      received
data04/bacula                      recordsize  16M      inherited from data04
data04/bacula/volumes              recordsize  16M      local
data04/bacula/volumes/DiffFile-03  recordsize  16M      inherited from data04/bacula/volumes
data04/bacula/volumes/FullFile-03  recordsize  16M      inherited from data04/bacula/volumes
data04/bacula/volumes/IncrFile-03  recordsize  16M      inherited from data04/bacula/volumes
[23:27 r730-01 dvl ~] % 

Let’s adjust compression too.

[23:27 r730-01 dvl ~] % zfs get -t filesystem -r compression data04
NAME                               PROPERTY     VALUE           SOURCE
data04                             compression  zstd            received
data04/bacula                      compression  zstd            inherited from data04
data04/bacula/volumes              compression  zstd            inherited from data04
data04/bacula/volumes/DiffFile-03  compression  zstd            received
data04/bacula/volumes/FullFile-03  compression  zstd            received
data04/bacula/volumes/IncrFile-03  compression  zstd            received

[23:28 r730-01 dvl ~] % sudo zfs set compression=zstd recordsize=128k data04

[23:29 r730-01 dvl ~] % zfs get -t filesystem -r recordsize data04          
NAME                               PROPERTY    VALUE    SOURCE
data04                             recordsize  128K     local
data04/bacula                      recordsize  128K     inherited from data04
data04/bacula/volumes              recordsize  16M      local
data04/bacula/volumes/DiffFile-03  recordsize  16M      inherited from data04/bacula/volumes
data04/bacula/volumes/FullFile-03  recordsize  16M      inherited from data04/bacula/volumes
data04/bacula/volumes/IncrFile-03  recordsize  16M      inherited from data04/bacula/volumes

[23:29 r730-01 dvl ~] % zfs get -t filesystem -r compression data04         
NAME                               PROPERTY     VALUE           SOURCE
data04                             compression  zstd            local
data04/bacula                      compression  zstd            inherited from data04
data04/bacula/volumes              compression  zstd            inherited from data04
data04/bacula/volumes/DiffFile-03  compression  zstd            received
data04/bacula/volumes/FullFile-03  compression  zstd            received
data04/bacula/volumes/IncrFile-03  compression  zstd            received

[23:29 r730-01 dvl ~] % sudo zfs inherit compression data04/bacula/volumes/IncrFile-03
[23:29 r730-01 dvl ~] % sudo zfs inherit compression data04/bacula/volumes/FullFile-03
[23:30 r730-01 dvl ~] % sudo zfs inherit compression data04/bacula/volumes/DiffFile-03

[23:30 r730-01 dvl ~] % zfs get -t filesystem -r compression data04                   
NAME                               PROPERTY     VALUE           SOURCE
data04                             compression  zstd            local
data04/bacula                      compression  zstd            inherited from data04
data04/bacula/volumes              compression  zstd            inherited from data04
data04/bacula/volumes/DiffFile-03  compression  zstd            inherited from data04
data04/bacula/volumes/FullFile-03  compression  zstd            inherited from data04
data04/bacula/volumes/IncrFile-03  compression  zstd            inherited from data04

Copy over the working directory

root@r730-01:~ # zfs snapshot data01/bacula/working@for-copying-to-data04

root@r730-01:~ # zfs send -Rv data01/bacula/working@for-data04-copy | zfs receive -uFv data04/bacula/working
skipping snapshot data01/bacula/working@for-copying-to-data04 because it was created after the destination snapshot (for-data04-copy)
full send of data01/bacula/working@for-data04-copy estimated size is 53.6K
total estimated size is 53.6K
TIME        SENT   SNAPSHOT data01/bacula/working@for-data04-copy
receiving full stream of data01/bacula/working@for-data04-copy into data04/bacula/working@for-data04-copy
received 81.7K stream in 0.14 seconds (570K/sec)
root@r730-01:~ # 

And there it is:

[23:43 r730-01 dvl ~] % zfs list -r data04/bacula                              
NAME                                USED  AVAIL  REFER  MOUNTPOINT
data04/bacula                      3.19T  17.4T   205K  /data04/bacula
data04/bacula/volumes              3.19T  17.4T   205K  /data04/bacula/volumes
data04/bacula/volumes/DiffFile-03   405G  17.4T   405G  /data04/bacula/volumes/DiffFile-03
data04/bacula/volumes/FullFile-03  2.45T  17.4T  2.44T  /data04/bacula/volumes/FullFile-03
data04/bacula/volumes/IncrFile-03   360G  17.4T   360G  /data04/bacula/volumes/IncrFile-03
data04/bacula/working               247K  17.4T   247K  /data04/bacula/working

Disable the old filesystems

First, get everything unmounted, by force, as is required.

[23:52 r730-01 dvl ~] % sudo zfs set canmount=off data01/bacula
cannot unmount '/jails/bacula-sd-03/usr/local/bacula/volumes': pool or dataset is busy
[23:52 r730-01 dvl ~] % sudo zfs umount data01/bacula/working
[23:53 r730-01 dvl ~] % sudo zfs umount data01/bacula/volumes/IncrFile-03
[23:53 r730-01 dvl ~] % sudo zfs umount data01/bacula/volumes/FullFile-03
[23:53 r730-01 dvl ~] % sudo zfs umount data01/bacula/volumes/DiffFile-03
[23:53 r730-01 dvl ~] % sudo zfs umount data01/bacula/volumes            


[0:06 r730-01 dvl ~] % mount | grep /jails/bacula-sd-03      
data02/jails/bacula-sd-03 on /jails/bacula-sd-03 (zfs, local, noatime, nfsv4acls)
data01/bacula on /jails/bacula-sd-03/usr/local/bacula (zfs, local, noatime, nfsv4acls)
data01/bacula/volumes on /jails/bacula-sd-03/usr/local/bacula/volumes (zfs, local, noatime, nfsv4acls)
[0:06 r730-01 dvl ~] % sudo zfs umount data01/bacula/volumes
cannot unmount '/jails/bacula-sd-03/usr/local/bacula/volumes': pool or dataset is busy
[0:08 r730-01 dvl ~] % sudo zfs umount -f data01/bacula/volumes
[0:08 r730-01 dvl ~] % sudo zfs umount -f data01/bacula        
[0:08 r730-01 dvl ~] % 


[0:09 r730-01 dvl ~] % sudo zfs set canmount=off data01/bacula       

That last bit makes sure it doesn’t get accidentally mounted when the system starts up.

And just for added protection, change it to something which will not interfere.

[0:10 r730-01 dvl ~] % sudo zfs set mountpoint=/data01/jails/bacula-sd-03/usr/local/bacula data01/bacula
[0:13 r730-01 dvl ~] % zfs get -r -t filesystem mountpoint data01/bacula                                
NAME                               PROPERTY    VALUE                                                            SOURCE
data01/bacula                      mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula                      local
data01/bacula/volumes              mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula/volumes              inherited from data01/bacula
data01/bacula/volumes/DiffFile-03  mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03  inherited from data01/bacula
data01/bacula/volumes/FullFile-03  mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03  inherited from data01/bacula
data01/bacula/volumes/IncrFile-03  mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03  inherited from data01/bacula
data01/bacula/working              mountpoint  /data01/jails/bacula-sd-03/usr/local/bacula/working              inherited from data01/bacula
[0:13 r730-01 dvl ~] % 

Set mountpoints for the new data

This is what we have now:

[0:13 r730-01 dvl ~] % zfs get -r -t filesystem mountpoint data04/bacula
NAME                               PROPERTY    VALUE                               SOURCE
data04/bacula                      mountpoint  /data04/bacula                      default
data04/bacula/volumes              mountpoint  /data04/bacula/volumes              default
data04/bacula/volumes/DiffFile-03  mountpoint  /data04/bacula/volumes/DiffFile-03  default
data04/bacula/volumes/FullFile-03  mountpoint  /data04/bacula/volumes/FullFile-03  default
data04/bacula/volumes/IncrFile-03  mountpoint  /data04/bacula/volumes/IncrFile-03  default
data04/bacula/working              mountpoint  /data04/bacula/working              default

Adjust that for use by the jail:

[0:14 r730-01 dvl ~] % sudo zfs set mountpoint=/jails/bacula-sd-03/usr/local/bacula data04/bacula

[0:15 r730-01 dvl ~] % zfs get canmount data04/bacula                                            
NAME           PROPERTY  VALUE     SOURCE
data04/bacula  canmount  on        default

[0:15 r730-01 dvl ~] % zfs get -r -t filesystem mountpoint data04/bacula                         
NAME                               PROPERTY    VALUE                                                     SOURCE
data04/bacula                      mountpoint  /jails/bacula-sd-03/usr/local/bacula                      local
data04/bacula/volumes              mountpoint  /jails/bacula-sd-03/usr/local/bacula/volumes              inherited from data04/bacula
data04/bacula/volumes/DiffFile-03  mountpoint  /jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03  inherited from data04/bacula
data04/bacula/volumes/FullFile-03  mountpoint  /jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03  inherited from data04/bacula
data04/bacula/volumes/IncrFile-03  mountpoint  /jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03  inherited from data04/bacula
data04/bacula/working              mountpoint  /jails/bacula-sd-03/usr/local/bacula/working              inherited from data04/bacula
[0:15 r730-01 dvl ~] % 

Verify they are mounted:

[0:15 r730-01 dvl ~] % zfs get -r -t filesystem mounted data04/bacula   
NAME                               PROPERTY  VALUE    SOURCE
data04/bacula                      mounted   yes      -
data04/bacula/volumes              mounted   yes      -
data04/bacula/volumes/DiffFile-03  mounted   yes      -
data04/bacula/volumes/FullFile-03  mounted   yes      -
data04/bacula/volumes/IncrFile-03  mounted   yes      -
data04/bacula/working              mounted   yes      -

Good to go.

Start the jail

OK, let’s see how this goes:

[0:17 r730-01 dvl ~] % sudo service jail start bacula-sd-03
Starting jails: bacula-sd-03.

And over here:

[19:18 pro05 dvl ~] % ssh bacula-sd-03
Last login: Mon Nov 10 00:40:07 2025 from pro04.startpoint.vpn.unixathome.org

[0:19 bacula-sd-03 dvl /usr/local/bacula/volumes] % ls -l
total 70
drwxr-xr-x  2 bacula bacula 224 2019.12.03 01:34 DiffFile-03/
drwxr-xr-x  2 bacula bacula 455 2022.11.07 14:21 FullFile-03/
drwxr-xr-x  2 bacula bacula 279 2023.11.16 03:47 IncrFile-03/
[0:19 bacula-sd-03 dvl /usr/local/bacula/volumes] % 

And it was now that I realized the working filesystem has a different mountpoint.

After a moment of thinking I had to modify something, I found this:

[0:25 bacula-sd-03 dvl /usr/local/bacula] % ls -l working
total 9
-rw-r-----  1 bacula bacula 2196 2025.12.09 20:24 bacula-sd.9103.state

All good.

The proper test comes after the next set of Full backups. They get copied over here. Now that the zpool has considerably more free space, I might start copying over incremental and differential backups as well.

Freeing up the space

I hate deleting data, even after copying it over and I know all is well.

It would be very easy: zfs destroy -r data01/bacula

It wouldn’t take very long. In fact, I did similar yesterday when I moved a zpool away. The old zpool was destroyed and I was OK with that. See data02 referenced in Making room to creating a raidz2 from 8x 4TB devices – here. we. go.

This has been me talking myself into it.

[0:33 r730-01 dvl ~] % zfs destroy -nrv data01/bacula
would destroy data01/bacula/working@for-copying-to-data04
would destroy data01/bacula/working@for-data04-copy
would destroy data01/bacula/working
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-07_00:00:04_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-23_00:00:09_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-29_00:00:02_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-20_00:00:03_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-17_00:00:01_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-21_00:00:09_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-30_00:00:03_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-01_00:00:01_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-26_00:00:02_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-04_00:00:08_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-16_00:00:16_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-27_00:00:04_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-25_00:00:10_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-28_00:00:02_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-10_00:00:01_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-24_00:00:01_daily
would destroy data01/bacula/volumes/FullFile-03@for-copy-to-data04
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-02_00:00:14_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-08_00:00:09_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-19_00:00:13_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-09_00:00:02_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-18_00:00:02_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-03_00:00:07_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-11-22_00:00:08_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-05_00:00:16_daily
would destroy data01/bacula/volumes/FullFile-03@autosnap_2025-12-06_00:00:13_daily
would destroy data01/bacula/volumes/FullFile-03
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-27_00:00:04_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-29_00:00:13_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-25_00:00:07_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-17_00:00:06_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-04_00:00:04_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-03_00:00:07_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-19_00:00:08_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-20_00:00:06_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-16_00:00:15_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-08_00:00:13_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-01_00:00:04_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-09_00:00:09_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-07_00:00:12_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-06_00:00:02_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-05_00:00:11_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-21_00:00:06_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-24_00:00:08_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-10_00:00:08_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-26_00:00:09_daily
would destroy data01/bacula/volumes/IncrFile-03@for-copy-to-data04
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-12-02_00:00:12_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-28_00:00:33_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-22_00:00:01_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-18_00:00:10_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-23_00:00:10_daily
would destroy data01/bacula/volumes/IncrFile-03@autosnap_2025-11-30_00:00:02_daily
would destroy data01/bacula/volumes/IncrFile-03
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-26_00:00:09_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-02_00:00:12_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-23_00:00:11_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-28_00:00:05_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-10_00:00:09_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-09_00:00:12_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-19_00:00:12_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-30_00:00:11_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-22_00:00:04_daily
would destroy data01/bacula/volumes/DiffFile-03@for-copy-to-data04
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-06_00:00:13_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-20_00:00:11_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-03_00:00:10_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-01_00:00:08_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-27_00:00:11_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-16_00:00:16_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-07_00:00:00_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-05_00:00:14_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-18_00:00:01_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-08_00:00:10_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-17_00:00:07_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-21_00:00:14_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-25_00:00:11_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-12-04_00:00:04_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-29_00:00:12_daily
would destroy data01/bacula/volumes/DiffFile-03@autosnap_2025-11-24_00:00:00_daily
would destroy data01/bacula/volumes/DiffFile-03
would destroy data01/bacula/volumes@for-copy-to-data04
would destroy data01/bacula/volumes
would destroy data01/bacula
[0:34 r730-01 dvl ~] % 

OK, here, we go:

[0:34 r730-01 dvl ~] % sudo zfs destroy -r data01/bacula
[0:35 r730-01 dvl ~] % zpool list
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  5.81T  3.56G  5.81T        -         -     0%     0%  1.00x    ONLINE  -
data02   928G   649G   279G        -         -     2%    69%  1.00x    ONLINE  -
data03  7.25T  1.29T  5.96T        -         -    33%    17%  1.00x    ONLINE  -
data04  29.1T  4.49T  24.6T        -         -     0%    15%  1.00x    ONLINE  -
zroot    107G  30.2G  76.8G        -         -    15%    28%  1.00x    ONLINE  -
[0:36 r730-01 dvl ~] % 

With that done, the zpool dropped below 80%

80% full on zpools?

A note about that: 80% is a good rule, but with lareger zpools, that value can be sent up considerably. On a 30TB zpool, I suspect it would be enough to have 1 or 2TB free. 80% full on a 29TB pool leaves about 6TB free – I think, without evidence, that 3 or 5% might be sufficient on a pool that large.

Top

creating a raidz2 from 8x 4TB devices – here. we. go.

Post by Dan Langille via Dan Langille's Other Diary »

It’s time.

Time to create the 8x 4T raidz2 zpool.

These are the devices: disks=”nda0 nda1 nda2 nda3 nda5 nda8 da12 da13″

In this post:

  • FreeBSD 14.3

Check for existing partitions

Using the list above:

[18:15 r730-01 dvl ~] % gpart show | grep -E "nda0|nda1|nda2|nda3|nda5|nda8|da12|da13"
=>        40  7814037088  nda5  GPT  (3.6T)
=>        34  7814037101  nda8  GPT  (3.6T)

Note I could not do this command because it fails at the first error:

[18:05 r730-01 dvl ~] % gpart show nda0 nda1 nda2 nda3 nda5 nda8 da12 da13
gpart: No such geom: nda0.

Looking through the output of gpart show -l, I find:

=>        40  7814037088  nda5  GPT  (3.6T)
          40  7814036000     1  Samsung_990_S7U8NJ0Y716854P  (3.6T)
  7814036040        1088        - free -  (544K)
..
=>        34  7814037101  nda8  GPT  (3.6T)
          34           6        - free -  (3.0K)
          40  7814036000     1  Samsung_990_S7U8NJ0Y716801F  (3.6T)
  7814036040        1095        - free -  (548K)

Remember that partition size (7814036000) as it will be referenced later.

Looking at the output of zpool import, I can confirm the above two devices can from the zpool formerly known as data02:

[18:18 r730-01 dvl ~] % sudo zpool import 
   pool: data02_old
     id: 14532602998618854058
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	data02_old                           ONLINE
	  mirror-0                           ONLINE
	    gpt/Samsung_990_S7U8NJ0Y716854P  ONLINE
	    gpt/Samsung_990_S7U8NJ0Y716801F  ONLINE

   pool: data04_old
     id: 14613959245391720618
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	data04_old  ONLINE
	  mirror-0  ONLINE
	    da12    ONLINE
	    da13    ONLINE

You should note that data04_old uses whole disks, no partitions. That’s not my usual approach, as I will soon demonstrate.

Clearing old partitions

This is me, clearing the partitions:

[18:56 r730-01 dvl ~] % sudo gpart destroy -F nda5
nda5 destroyed
[18:57 r730-01 dvl ~] % sudo gpart destroy -F nda8
nda8 destroyed

That takes care of data02_old.

You will notice the timestamp in the command prompt is not in sequence. I found I had to redo some of this work.

Clearing old labels

I use these commands to clear out the existing labels. This is a precaution, just because I can.

[18:26 r730-01 dvl ~] % sudo zpool labelclear da12  
use '-f' to override the following error:
/dev/da12 is a member of exported pool "data04_old"
[18:26 r730-01 dvl ~] % sudo zpool labelclear -f da12
[18:26 r730-01 dvl ~] % sudo zpool labelclear -f da13

That cleared out data04_old – it’s gone.

Now there’s nothing to import:

[18:29 r730-01 dvl ~] % sudo zpool import
no pools available to import

All the diskinfo

Scripting is good, it saves us time:

[18:32 r730-01 dvl ~] % echo "nda0 nda1 nda2 nda3 nda5 nda8 da12 da13" | xargs -n 1 -I % sudo diskinfo -v /dev/%
/dev/nda0
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 PRO 4TB	# Disk descr.
	S7KGNU0Y722875X	# Disk ident.
	nvme0       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/nda1
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 PRO 4TB	# Disk descr.
	S7KGNU0Y915666E	# Disk ident.
	nvme1       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/nda2
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 PRO 4TB	# Disk descr.
	S7KGNU0Y912937J	# Disk ident.
	nvme2       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/nda3
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 PRO 4TB	# Disk descr.
	S7KGNU0Y912955D	# Disk ident.
	nvme3       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/nda5
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 EVO Plus 4TB	# Disk descr.
	S7U8NJ0Y716854P	# Disk ident.
	nvme5       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/nda8
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 990 EVO Plus 4TB	# Disk descr.
	S7U8NJ0Y716801F	# Disk ident.
	nvme8       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

/dev/da12
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	4096        	# stripesize
	0           	# stripeoffset
	486401      	# Cylinders according to firmware.
	255         	# Heads according to firmware.
	63          	# Sectors according to firmware.
	ATA Samsung SSD 870	# Disk descr.
	S757NS0Y700758M	# Disk ident.
	mrsas0      	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM
	Not_Zoned   	# Zone Mode

/dev/da13
	512         	# sectorsize
	4000787030016	# mediasize in bytes (3.6T)
	7814037168  	# mediasize in sectors
	4096        	# stripesize
	0           	# stripeoffset
	486401      	# Cylinders according to firmware.
	255         	# Heads according to firmware.
	63          	# Sectors according to firmware.
	ATA Samsung SSD 870	# Disk descr.
	S757NS0Y700760R	# Disk ident.
	mrsas0      	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM
	Not_Zoned   	# Zone Mode

[18:32 r730-01 dvl ~] % 

Creating new partitioning schemes

This is the start of our partitions, or more precisely, our partition schemes into which we will add our partition.

[18:40 r730-01 dvl ~] % echo "nda0 nda1 nda2 nda3 nda5 nda8 da12 da13" | xargs -n 1 -I % sudo gpart create -s gpt %
nda0 created
nda1 created
nda2 created
nda3 created
nda5 created
nda8 created
da12 created
da13 created

Creating the freebsd-zfs partitions

With the diskinfo output (shown above) in one terminal, it was easy to do these commands. Again, ignoring those two messages is OK, it is expected.

[18:56 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7KGNU0Y722875X nda0
nda0p1 added
[19:01 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7KGNU0Y915666E nda1
nda1p1 added
[19:02 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7KGNU0Y912937J nda2
nda2p1 added
[19:02 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7KGNU0Y912955D nda3
nda3p1 added
[19:02 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7U8NJ0Y716854P nda5
nda5p1 added
[19:02 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S7U8NJ0Y716801F nda8
nda8p1 added
[19:03 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S757NS0Y700758M da12
da12p1 added
[19:03 r730-01 dvl ~] % sudo gpart add -a 4K -i 1 -t freebsd-zfs -s 7814036000 -l S757NS0Y700760R da13
da13p1 added
[19:03 r730-01 dvl ~] % 

Checking the partition sizes

Let’s check the partitions using the same command which failed earlier:

[19:01 r730-01 dvl ~] % gpart show nda0 nda1 nda2 nda3 nda5 nda8 da12 da13
=>        40  7814037088  nda0  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda1  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda2  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda3  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda5  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  nda8  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  da12  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

=>        40  7814037088  da13  GPT  (3.6T)
          40  7814036000     1  freebsd-zfs  (3.6T)
  7814036040        1088        - free -  (544K)

Creating the zpool

Here we go, creating the big zpool. I’m basing this on the one done in prod – moving to a SATADOM based zpool (zroot) using zfs snapshot and send | recv (more or less).

zpool create \
  -O mountpoint=none \
  -O atime=off \
  -O compression=zstd \
data04 raidz2 /dev/gpt/S7KGNU0Y722875X /dev/gpt/S7KGNU0Y915666E /dev/gpt/S7KGNU0Y912937J /dev/gpt/S7KGNU0Y912955D /dev/gpt/S7U8NJ0Y716854P /dev/gpt/S7U8NJ0Y716801F /dev/gpt/S757NS0Y700758M /dev/gpt/S757NS0Y700760R

To avoid copy/paste, I ran this command to get the list of devices:

[19:09 r730-01 dvl ~] % gpart show -l nda0 nda1 nda2 nda3 nda5 nda8 da12 da13 | grep 7814036000 | cut -f 5 -w | xargs -n 1 -I % echo /dev/gpt/% | xargs
/dev/gpt/S7KGNU0Y722875X /dev/gpt/S7KGNU0Y915666E /dev/gpt/S7KGNU0Y912937J /dev/gpt/S7KGNU0Y912955D /dev/gpt/S7U8NJ0Y716854P /dev/gpt/S7U8NJ0Y716801F /dev/gpt/S757NS0Y700758M /dev/gpt/S757NS0Y700760R

It exploits the fact that each partition has size 7814036000.

Creation:

[19:09 r730-01 dvl ~] % sudo zpool create \
  -O mountpoint=none \
  -O atime=off \
  -O compression=zstd \
data04 raidz2 /dev/gpt/S7KGNU0Y722875X /dev/gpt/S7KGNU0Y915666E /dev/gpt/S7KGNU0Y912937J /dev/gpt/S7KGNU0Y912955D /dev/gpt/S7U8NJ0Y716854P /dev/gpt/S7U8NJ0Y716801F /dev/gpt/S757NS0Y700758M /dev/gpt/S757NS0Y700760R

[19:12 r730-01 dvl ~] % zpool list
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  5.81T  5.10T   731G        -         -     3%    87%  1.00x    ONLINE  -
data02   928G   648G   280G        -         -     0%    69%  1.00x    ONLINE  -
data03  7.25T  1.29T  5.96T        -         -    33%    17%  1.00x    ONLINE  -
data04  29.1T  1.34M  29.1T        -         -     0%     0%  1.00x    ONLINE  -
zroot    107G  30.2G  76.8G        -         -    15%    28%  1.00x    ONLINE  -

[19:12 r730-01 dvl ~] % zpool status data04
  pool: data04
 state: ONLINE
config:

	NAME                     STATE     READ WRITE CKSUM
	data04                   ONLINE       0     0     0
	  raidz2-0               ONLINE       0     0     0
	    gpt/S7KGNU0Y722875X  ONLINE       0     0     0
	    gpt/S7KGNU0Y915666E  ONLINE       0     0     0
	    gpt/S7KGNU0Y912937J  ONLINE       0     0     0
	    gpt/S7KGNU0Y912955D  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716854P  ONLINE       0     0     0
	    gpt/S7U8NJ0Y716801F  ONLINE       0     0     0
	    gpt/S757NS0Y700758M  ONLINE       0     0     0
	    gpt/S757NS0Y700760R  ONLINE       0     0     0

errors: No known data errors

I figured 8 x 4T in raidz2 would give me a 24T pool. I seem to have been off by 5T.

Top

Making room to creating a raidz2 from 8x 4TB devices – here. we. go.

Post by Dan Langille via Dan Langille's Other Diary »

It’s time.

Time to create the 8x 4T raidz2 zpool. That doesn’t happen in this post, hopefully the next one.

But first, I have to move some stuff around, to free up all the 4TB devices.

The host has 4 devices (NVMe) and I moved in two more, 4TB SSDs. From /var/log/messages:

Dec  9 00:24:29 r730-01 kernel: mrsas0: System PD created target ID: 0xc
Dec  9 00:24:29 r730-01 kernel: mrsas0: System PD created target ID: 0xd
Dec  9 00:24:30 r730-01 kernel: da12 at mrsas0 bus 1 scbus1 target 13 lun 0
Dec  9 00:24:30 r730-01 kernel: da12:  Fixed Direct Access SPC-4 SCSI device
Dec  9 00:24:30 r730-01 kernel: da12: Serial Number S757NS0Y700760R
Dec  9 00:24:30 r730-01 kernel: da12: 150.000MB/s transfers
Dec  9 00:24:30 r730-01 kernel: da12: 3815447MB (7814037168 512 byte sectors)
Dec  9 00:24:30 r730-01 kernel: da12: quirks=0x8<4K>
Dec  9 00:24:30 r730-01 kernel: da13 at mrsas0 bus 1 scbus1 target 12 lun 0
Dec  9 00:24:30 r730-01 kernel: da13:  Fixed Direct Access SPC-4 SCSI device
Dec  9 00:24:30 r730-01 kernel: da13: Serial Number S757NS0Y700758M
Dec  9 00:24:30 r730-01 kernel: da13: 150.000MB/s transfers
Dec  9 00:24:30 r730-01 kernel: da13: 3815447MB (7814037168 512 byte sectors)
Dec  9 00:24:30 r730-01 kernel: da13: quirks=0x8<4K>

That makes the list now: disks=”nda0 nda1 nda2 nda3 nda5 nda8 da12 da13″

Oh, but two of those disks are involved in an exist zpool: data02

[0:36 r730-01 dvl ~/tmp] % zpool list       
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  5.81T  5.10T   731G        -         -     3%    87%  1.00x    ONLINE  -
data02  3.62T   688G  2.95T        -         -    33%    18%  1.00x    ONLINE  -
data03  7.25T  1.28T  5.97T        -         -    32%    17%  1.00x    ONLINE  -
zroot    107G  30.1G  76.9G        -         -    15%    28%  1.00x    ONLINE  -

It has 688G on it. However, it’s the main jail zpool.

[0:50 r730-01 dvl ~/tmp] % zfs list -d 2 -r data02
NAME                                USED  AVAIL  REFER  MOUNTPOINT
data02                              868G  2.66T    96K  none
data02/freshports                   307G  2.66T    88K  none
data02/freshports/dev-ingress01     244G  2.66T    88K  none
data02/freshports/dev-nginx01      55.9M  2.66T    96K  none
data02/freshports/dvl-ingress01    18.6G  2.66T    96K  none
data02/freshports/dvl-nginx01      23.4M  2.66T    96K  none
data02/freshports/jailed            449M  2.66T    96K  none
data02/freshports/stage-ingress01  19.6G  2.66T    96K  none
data02/freshports/test-ingress01   24.2G  2.66T    96K  none
data02/jails                        309G  2.66T  9.54G  /jails
data02/jails/bacula                16.9G  2.66T  16.3G  /jails/bacula
data02/jails/bacula-sd-02          3.45G  2.66T  2.84G  /jails/bacula-sd-02
data02/jails/bacula-sd-03          4.85G  2.66T  4.25G  /jails/bacula-sd-03
data02/jails/besser                5.63G  2.66T  4.21G  /jails/besser
data02/jails/certs                 3.02G  2.66T  2.42G  /jails/certs
data02/jails/certs-rsync           3.00G  2.66T  2.42G  /jails/certs-rsync
data02/jails/cliff2                3.05G  2.66T  2.47G  /jails/cliff2
data02/jails/dev-ingress01         4.62G  2.66T  3.59G  /jails/dev-ingress01
data02/jails/dev-nginx01           4.47G  2.66T  3.44G  /jails/dev-nginx01
data02/jails/dev-pgeu              7.62G  2.66T  7.62G  /jails/dev-pgeu
data02/jails/dns-hidden-master     3.21G  2.66T  2.57G  /jails/dns-hidden-master
data02/jails/dns1                  11.0G  2.66T  4.95G  /jails/dns1
data02/jails/dvl-ingress01         7.81G  2.66T  5.28G  /jails/dvl-ingress01
data02/jails/dvl-nginx01           2.15G  2.66T  1.17G  /jails/dvl-nginx01
data02/jails/freebsd143             501M  2.66T   501M  /jails/freebsd143
data02/jails/git                   4.94G  2.66T  4.34G  /jails/git
data02/jails/jail_within_jail      1.28G  2.66T   750M  /jails/jail_within_jail
data02/jails/mqtt01                3.60G  2.66T  3.00G  /jails/mqtt01
data02/jails/mydev                 26.0G  2.66T  22.0G  /jails/mydev
data02/jails/mysql01               17.4G  2.66T  7.77G  /jails/mysql01
data02/jails/nsnotify              3.28G  2.66T  2.61G  /jails/nsnotify
data02/jails/pg01                  46.0G  2.66T  10.2G  /jails/pg01
data02/jails/pg02                  11.0G  2.66T  10.4G  /jails/pg02
data02/jails/pg03                  11.3G  2.66T  9.41G  /jails/pg03
data02/jails/pkg01                 16.9G  2.66T  13.8G  /jails/pkg01
data02/jails/samdrucker            4.88G  2.66T  4.27G  /jails/samdrucker
data02/jails/serpico               3.10G  2.66T  2.49G  /jails/serpico
data02/jails/stage-ingress01       5.21G  2.66T  2.89G  /jails/stage-ingress01
data02/jails/stage-nginx01         2.44G  2.66T  1.38G  /jails/stage-nginx01
data02/jails/svn                   10.5G  2.66T  9.77G  /jails/svn
data02/jails/talos                 2.98G  2.66T  2.40G  /jails/talos
data02/jails/test-ingress01        2.31G  2.66T  1.35G  /jails/test-ingress01
data02/jails/test-nginx01          2.38G  2.66T  1.30G  /jails/test-nginx01
data02/jails/unifi01               30.2G  2.66T  12.2G  /jails/unifi01
data02/jails/webserver             12.7G  2.66T  11.5G  /jails/webserver
data02/reserved                     180G  2.84T    96K  none
data02/vm                          70.8G  2.66T  7.24G  /usr/local/vm
data02/vm/freebsd-test              743M  2.66T   112K  /usr/local/vm/freebsd-test
data02/vm/hass                     59.1G  2.66T  13.4G  /usr/local/vm/hass
data02/vm/home-assistant            351M  2.66T   351M  /usr/local/vm/home-assistant
data02/vm/myguest                  3.38G  2.66T  3.38G  /usr/local/vm/myguest

It seems I should move this elsewhere first. I know nda6 and nda7 are 1TB.

[0:31 r730-01 dvl ~/tmp] % sudo diskinfo -v /dev/nda6


/dev/nda6
	512         	# sectorsize
	1000204886016	# mediasize in bytes (932G)
	1953525168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 980 PRO with Heatsink 1TB	# Disk descr.
	S6WSNJ0T208743F	# Disk ident.
	nvme6       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

[0:31 r730-01 dvl ~/tmp] % sudo diskinfo -v /dev/nda7
/dev/nda7
	512         	# sectorsize
	1000204886016	# mediasize in bytes (932G)
	1953525168  	# mediasize in sectors
	0           	# stripesize
	0           	# stripeoffset
	Samsung SSD 980 PRO with Heatsink 1TB	# Disk descr.
	S6WSNJ0T207774T	# Disk ident.
	nvme7       	# Attachment
	Yes         	# TRIM/UNMAP support
	0           	# Rotation rate in RPM

I create the new zpool:

[0:54 r730-01 dvl ~] % sudo gpart create -s gpt  nda6
nda6 created
[0:54 r730-01 dvl ~] % sudo gpart create -s gpt  nda7
nda7 created

[0:56 r730-01 dvl ~] % sudo gpart add -i 1 -t freebsd-zfs -l S6WSNJ0T208743F nda6
nda6p1 added
[0:56 r730-01 dvl ~] % sudo gpart add -i 1 -t freebsd-zfs -l S6WSNJ0T207774T  nda7
nda7p1 added

[0:56 r730-01 dvl ~] % sudo zpool create data02_new mirror /dev/gpt/S6WSNJ0T208743F /dev/gpt/S6WSNJ0T207774T
[0:57 r730-01 dvl ~] % 

[0:57 r730-01 dvl ~] % zpool list data02_new  
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data02_new   928G   372K   928G        -         -     0%     0%  1.00x    ONLINE  -
[0:57 r730-01 dvl ~] % zpool status data02_new
  pool: data02_new
 state: ONLINE
config:

	NAME                     STATE     READ WRITE CKSUM
	data02_new               ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    gpt/S6WSNJ0T208743F  ONLINE       0     0     0
	    gpt/S6WSNJ0T207774T  ONLINE       0     0     0

errors: No known data errors
[0:58 r730-01 dvl ~] % sudo zpool scrub data02_new
[0:58 r730-01 dvl ~] % gpart show nda6 nda7
=>        40  1953525088  nda6  GPT  (932G)
          40  1953525088     1  freebsd-zfs  (932G)

=>        40  1953525088  nda7  GPT  (932G)
          40  1953525088     1  freebsd-zfs  (932G)

[0:58 r730-01 dvl ~] % 

I ran the scrub so that my Nagios monitoring does not warn me there is no recent scrub on that zpool.

Copying the data

Now I want to copy all the data from data02 into data02_new so I can repurpose those drives into an 8x 4T zpool array.

I’m heading to bed. I’ll come back to this another day.

Tuesday – the initial send | recv

I do a snapshot, then send | receive. I thought about doing this from a live-boot off a thumb drive.

Or, I could just do it now:

[16:24 r730-01 dvl ~] % sudo service jail stop 
Stopping jails: unifi01 nsnotify dns-hidden-master serpico jail_within_jail mydev bacula-sd-03 samdrucker talos bacula-sd-02 mqtt01 webserver svn git certs certs-rsync besser bacula stage-nginx01 stage-ingress01 test-nginx01 test-ingress01 dvl-nginx01 dvl-ingress01 dev-nginx01 dev-ingress01 pkg01 pg03 pg02 pg01 mysql01 cliff2 dns1.

[16:26 r730-01 dvl ~] % sudo vm list
NAME            DATASTORE  LOADER     CPU  MEMORY  VNC  AUTO     STATE
freebsd-test    default    bhyveload  1    256M    -    No       Stopped
hass            default    uefi       4    8GB     -    Yes [1]  Running (2133)
home-assistant  default    uefi       1    1GB     -    No       Stopped
myguest         default    bhyveload  1    768M    -    No       Stopped

[16:26 r730-01 dvl ~] % sudo vm stop hass
Sending ACPI shutdown to hass
[16:26 r730-01 dvl ~] % 

Here goes the snapshot and the send | recv:

root@r730-01:/home/dvl # zfs  snapshot -r data02@for-transfer-to-data02_new
root@r730-01:/home/dvl # time zfs send -Rv data02@for-transfer-to-data02_new | zfs receive -uFv data02_new
full send of data02@delete-later-full2 estimated size is 43.1K
send from @delete-later-full2 to data02@delete-later-full3 estimated size is 624B
send from @delete-later-full3 to data02@for-transfer-to-data02_new estimated size is 624B
full send of data02/reserved@delete-later-full2 estimated size is 43.1K
send from @delete-later-full2 to data02/reserved@delete-later-full3 estimated size is 624B
send from @delete-later-full3 to data02/reserved@for-transfer-to-data02_new estimated size is 624B
full send of data02/freshports@delete-later-full2 estimated size is 39.3K
send from @delete-later-full2 to data02/freshports@delete-later-full3 estimated size is 624B
send from @delete-later-full3 to data02/freshports@for-transfer-to-data02_new estimated size is 624B
full send of data02/freshports/dev-ingress01@delete-later-full2 estimated size is 39.3K
send from @delete-later-full2 to data02/freshports/dev-ingress01@delete-later-full3 estimated size is 624B

...

TIME        SENT   SNAPSHOT data02/vm/hass@for-transfer-to-data02_new
received 248M stream in 0.45 seconds (553M/sec)
receiving incremental stream of data02/vm/hass@autosnap_2025-12-09_16:30:02_frequently into data02_new/vm/hass@autosnap_2025-12-09_16:30:02_frequently
received 312B stream in 0.02 seconds (13.0K/sec)
receiving incremental stream of data02/vm/hass@for-transfer-to-data02_new into data02_new/vm/hass@for-transfer-to-data02_new
received 312B stream in 0.02 seconds (13.2K/sec)
     1393.06 real         0.76 user       847.86 sys
root@r730-01:/home/dvl # 

A whole lot of export attempts and umounts

The following issues are a good view into why jails should mount and umount everything they need on start and stop, respectively. If I had that in place, the following would not have taken so long:

root@r730-01:/home/dvl # zpool export data02
cannot unmount '/jails/svn': pool or dataset is busy
root@r730-01:/home/dvl # mount | grep svn
data02/jails/svn on /jails/svn (zfs, local, noatime, nfsv4acls)
data03/repos/subversion on /jails/svn/usr/local/svn/repos (zfs, local, noatime, nfsv4acls)
root@r730-01:/home/dvl # zfs umount data03/repos/subversion
root@r730-01:/home/dvl # zpool export data02
cannot unmount '/jails/pkg01': pool or dataset is busy
root@r730-01:/home/dvl # mount | grep pkg01
data02/jails/pkg01 on /jails/pkg01 (zfs, local, noatime, nfsv4acls)
data03/poudriere/ccache on /jails/pkg01/var/ccache (zfs, local, noatime, nfsv4acls)
data03/poudriere/distfiles on /jails/pkg01/usr/ports/distfiles (zfs, local, noatime, nfsv4acls)
data03/poudriere on /jails/pkg01/usr/local/poudriere (zfs, local, noatime, nfsv4acls)
data03/poudriere/data on /jails/pkg01/usr/local/poudriere/data (zfs, local, noatime, nfsv4acls)
data03/poudriere/jails on /jails/pkg01/usr/local/poudriere/jails (zfs, local, noatime, nfsv4acls)
data03/poudriere/test on /jails/pkg01/usr/local/poudriere/test (zfs, local, noatime, nfsv4acls)
data03/poudriere/ports on /jails/pkg01/usr/local/poudriere/ports (zfs, local, noatime, nfsv4acls)
data03/poudriere/jails/143amd64 on /jails/pkg01/usr/local/poudriere/jails/143amd64 (zfs, local, noatime, nfsv4acls)
data03/poudriere/data/cronjob-logs on /jails/pkg01/usr/local/poudriere/data/cronjob-logs (zfs, local, noatime, nfsv4acls)
data03/poudriere/data/cache on /jails/pkg01/usr/local/poudriere/data/cache (zfs, local, noatime, nfsv4acls)
data03/poudriere/data/packages on /jails/pkg01/usr/local/poudriere/data/packages (zfs, local, noatime, nfsv4acls)
data03/poudriere/jails/142amd64 on /jails/pkg01/usr/local/poudriere/jails/142amd64 (zfs, local, noatime, nfsv4acls)
data03/poudriere/ports/default on /jails/pkg01/usr/local/poudriere/ports/default (zfs, local, noatime, nfsv4acls)
data03/poudriere/ports/testing on /jails/pkg01/usr/local/poudriere/ports/testing (zfs, local, noatime, nfsv4acls)
data03/poudriere/ports/main on /jails/pkg01/usr/local/poudriere/ports/main (zfs, local, noatime, nfsv4acls)
data03/poudriere/ports/pgeu_system on /jails/pkg01/usr/local/poudriere/ports/pgeu_system (zfs, local, noatime, nfsv4acls)
root@r730-01:/home/dvl # zfs list -r data03
NAME                                       USED  AVAIL  REFER  MOUNTPOINT
data03                                    1.64T  5.48T    96K  none
data03/acme-certs                         11.9M  5.48T    96K  none
data03/acme-certs/certs                   6.58M  5.48T  5.67M  /jails/certs/var/db/acme/certs
data03/acme-certs/certs-for-rsync         5.26M  5.48T  4.03M  /jails/certs/var/db/certs-for-rsync
data03/dvl                                 120K  5.48T   120K  /jails/dev-ingress01/usr/home/dvl/src
data03/jail_within_jail                   1.08G  5.48T    96K  none
data03/jail_within_jail/jails             1.08G  5.48T    96K  none
data03/jail_within_jail/jails/freshports  1.08G  5.48T  1.08G  none
data03/librenms-rrd                        117G  5.48T  25.5G  /jails/besser/var/db/librenms/rrd
data03/pg01                               71.2G  5.48T    96K  none
data03/pg01/freshports.dvl                37.1G  5.48T  27.6G  /jails/pg01/var/db/postgres.freshports.dvl
data03/pg01/postgres                      34.1G  5.48T  27.6G  /jails/pg01/var/db/postgres
data03/pg02                               78.5G  5.48T    88K  none
data03/pg02/postgres                      78.5G  5.48T  51.8G  /jails/pg02/var/db/postgres
data03/pg02/rsyncer                       1.02M  5.48T   144K  /jails/pg02/usr/home/rsyncer/backups
data03/pg03                                772G  5.48T    88K  none
data03/pg03/postgres                       573G  5.48T   550G  /jails/pg03/var/db/postgres
data03/pg03/rsyncer                        199G  5.48T  33.2G  /jails/pg03/usr/home/rsyncer/backups
data03/poudriere                           245G  5.48T    96K  /usr/local/poudriere
data03/poudriere/ccache                   16.5M  5.48T   104K  /var/ccache
data03/poudriere/ccache/ccache.13amd64      96K  5.48T    96K  /var/ccache/ccache.13amd64
data03/poudriere/ccache/ccache.amd64      16.3M  5.48T  16.3M  /var/ccache/ccache.amd64
data03/poudriere/data                     93.1G  5.48T  28.1G  /usr/local/poudriere/data
data03/poudriere/data/cache                211M  5.48T   174M  /usr/local/poudriere/data/cache
data03/poudriere/data/cronjob-logs        4.01M  5.48T  1.61M  /usr/local/poudriere/data/cronjob-logs
data03/poudriere/data/packages            64.6G  5.48T  59.0G  /usr/local/poudriere/data/packages
data03/poudriere/distfiles                 132G  5.48T   132G  /usr/ports/distfiles
data03/poudriere/jails                    2.75G  5.48T    88K  /usr/local/poudriere/jails
data03/poudriere/jails/142amd64           1.37G  5.48T  1.37G  /usr/local/poudriere/jails/142amd64
data03/poudriere/jails/143amd64           1.38G  5.48T  1.38G  /usr/local/poudriere/jails/143amd64
data03/poudriere/ports                    16.4G  5.48T  3.25G  /usr/local/poudriere/ports
data03/poudriere/ports/default            2.97G  5.48T  2.77G  /usr/local/poudriere/ports/default
data03/poudriere/ports/main               1.60G  5.48T  1.60G  /usr/local/poudriere/ports/main
data03/poudriere/ports/pgeu_system        1.06G  5.48T  1.06G  /usr/local/poudriere/ports/pgeu_system
data03/poudriere/ports/testing            7.02G  5.48T  2.71G  /usr/local/poudriere/ports/testing
data03/poudriere/test                       96K  5.48T    96K  /usr/local/poudriere/test
data03/public                             23.4G  5.48T  23.4G  none
data03/repos                              9.05G  5.48T    88K  none
data03/repos/gitea                         812M  5.48T   308M  /jails/git/var/db/gitea
data03/repos/subversion                   8.25G  5.48T  7.75G  /jails/svn/usr/local/svn/repos
data03/reserved                            362G  5.84T    96K  none
root@r730-01:/home/dvl # zpool export data03
root@r730-01:/home/dvl # zpool export data02
cannot unmount '/jails/dev-ingress01/jails/freshports': pool or dataset is busy
root@r730-01:/home/dvl # mount | grep /jails/dev-ingress01/jails/freshports
data02/freshports/dev-ingress01/jails/freshports on /jails/dev-ingress01/jails/freshports (zfs, local, noatime, nfsv4acls)
devfs on /jails/dev-ingress01/jails/freshports/dev (devfs)
root@r730-01:/home/dvl # umount /jails/dev-ingress01/jails/freshports/dev
root@r730-01:/home/dvl # mount | grep /dev
devfs on /dev (devfs)
data02/jails/dev-ingress01 on /jails/dev-ingress01 (zfs, local, noatime, nfsv4acls)
data02/freshports/dev-ingress01/jails on /jails/dev-ingress01/jails (zfs, local, noatime, nfsv4acls)
data02/freshports/dev-ingress01/jails/freshports on /jails/dev-ingress01/jails/freshports (zfs, local, noatime, nfsv4acls)
root@r730-01:/home/dvl # zpool export data02
cannot unmount '/jails/bacula-sd-03': pool or dataset is busy
root@r730-01:/home/dvl # mount | grep /jails/bacula-sd-03
data02/jails/bacula-sd-03 on /jails/bacula-sd-03 (zfs, local, noatime, nfsv4acls)
data01/bacula on /jails/bacula-sd-03/usr/local/bacula (zfs, local, noatime, nfsv4acls)
data01/bacula/working on /jails/bacula-sd-03/usr/local/bacula/working (zfs, local, noatime, nfsv4acls)
data01/bacula/volumes on /jails/bacula-sd-03/usr/local/bacula/volumes (zfs, local, noatime, nfsv4acls)
data01/bacula/volumes/DiffFile-03 on /jails/bacula-sd-03/usr/local/bacula/volumes/DiffFile-03 (zfs, local, noatime, nfsv4acls)
data01/bacula/volumes/IncrFile-03 on /jails/bacula-sd-03/usr/local/bacula/volumes/IncrFile-03 (zfs, local, noatime, nfsv4acls)
data01/bacula/volumes/FullFile-03 on /jails/bacula-sd-03/usr/local/bacula/volumes/FullFile-03 (zfs, local, noatime, nfsv4acls)
root@r730-01:/home/dvl # zpool export data01
root@r730-01:/home/dvl # zpool export data02
root@r730-01:/home/dvl # 

Renaming

This renames data02:

root@r730-01:/home/dvl # zpool import data02 data02_old
root@r730-01:/home/dvl # zpool export data02_old
root@r730-01:/home/dvl # 

Now let’s check mount points:

root@r730-01:/home/dvl # zfs get -t filesystem -r mountpoint data02
NAME                                                                PROPERTY    VALUE                                                            SOURCE
data02                                                              mountpoint  none                                                             received
data02/freshports                                                   mountpoint  none                                                             inherited from data02
data02/freshports/dev-ingress01                                     mountpoint  none                                                             received
data02/freshports/dev-ingress01/dvl-src                             mountpoint  /jails/dev-ingress01/usr/home/dvl/src                            received
data02/freshports/dev-ingress01/freshports                          mountpoint  /jails/dev-ingress01/var/db/freshports                           received
data02/freshports/dev-ingress01/freshports/cache                    mountpoint  /jails/dev-ingress01/var/db/freshports/cache                     inherited from data02/freshports/dev-ingress01/freshports
data02/freshports/dev-ingress01/freshports/cache/html               mountpoint  /jails/dev-ingress01/var/db/freshports/cache/html                inherited from data02/freshports/dev-ingress01/freshports
data02/freshports/dev-ingress01/freshports/cache/spooling           mountpoint  /jails/dev-ingress01/var/db/freshports/cache/spooling            inherited from data02/freshports/dev-ingress01/freshports
data02/freshports/dev-ingress01/freshports/message-queues           mountpoint  /jails/dev-ingress01/var/db/freshports/message-queues            received
data02/freshports/dev-ingress01/freshports/message-queues/archive   mountpoint  /jails/dev-ingress01/var/db/freshports/message-queues/archive    inherited from data02/freshports/dev-ingress01/freshports/message-queues
data02/freshports/dev-ingress01/ingress                             mountpoint  /jails/dev-ingress01/var/db/ingress                              received
data02/freshports/dev-ingress01/ingress/latest_commits              mountpoint  /jails/dev-ingress01/var/db/ingress/latest_commits               inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/ingress/message-queues              mountpoint  /jails/dev-ingress01/var/db/ingress/message-queues               inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/ingress/repos                       mountpoint  /jails/dev-ingress01/var/db/ingress/repos                        inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/ingress/repos/doc                   mountpoint  /jails/dev-ingress01/var/db/ingress/repos/doc                    inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/ingress/repos/ports                 mountpoint  /jails/dev-ingress01/var/db/ingress/repos/ports                  inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/ingress/repos/src                   mountpoint  /jails/dev-ingress01/var/db/ingress/repos/src                    inherited from data02/freshports/dev-ingress01/ingress
data02/freshports/dev-ingress01/jails                               mountpoint  /jails/dev-ingress01/jails                                       received
data02/freshports/dev-ingress01/jails/freshports                    mountpoint  /jails/dev-ingress01/jails/freshports                            inherited from data02/freshports/dev-ingress01/jails
data02/freshports/dev-ingress01/jails/freshports/ports              mountpoint  /jails/dev-ingress01/jails/freshports/usr/ports                  received
data02/freshports/dev-ingress01/modules                             mountpoint  /jails/dev-ingress01/usr/local/lib/perl5/site_perl/FreshPorts    received
data02/freshports/dev-ingress01/scripts                             mountpoint  /jails/dev-ingress01/usr/local/libexec/freshports                received
data02/freshports/dev-nginx01                                       mountpoint  none                                                             inherited from data02
data02/freshports/dev-nginx01/www                                   mountpoint  /jails/dev-nginx01/usr/local/www                                 received
data02/freshports/dev-nginx01/www/freshports                        mountpoint  /jails/dev-nginx01/usr/local/www/freshports                      inherited from data02/freshports/dev-nginx01/www
data02/freshports/dev-nginx01/www/freshsource                       mountpoint  /jails/dev-nginx01/usr/local/www/freshsource                     inherited from data02/freshports/dev-nginx01/www
data02/freshports/dvl-ingress01                                     mountpoint  none                                                             received
data02/freshports/dvl-ingress01/dvl-src                             mountpoint  /jails/dvl-ingress01/usr/home/dvl/src                            received
data02/freshports/dvl-ingress01/freshports                          mountpoint  /jails/dvl-ingress01/var/db/freshports                           received
data02/freshports/dvl-ingress01/freshports/cache                    mountpoint  /jails/dvl-ingress01/var/db/freshports/cache                     inherited from data02/freshports/dvl-ingress01/freshports
data02/freshports/dvl-ingress01/freshports/cache/html               mountpoint  /jails/dvl-ingress01/var/db/freshports/cache/html                inherited from data02/freshports/dvl-ingress01/freshports
data02/freshports/dvl-ingress01/freshports/cache/spooling           mountpoint  /jails/dvl-ingress01/var/db/freshports/cache/spooling            inherited from data02/freshports/dvl-ingress01/freshports
data02/freshports/dvl-ingress01/freshports/message-queues           mountpoint  /jails/dvl-ingress01/var/db/freshports/message-queues            inherited from data02/freshports/dvl-ingress01/freshports
data02/freshports/dvl-ingress01/freshports/message-queues/archive   mountpoint  /jails/dvl-ingress01/var/db/freshports/message-queues/archive    inherited from data02/freshports/dvl-ingress01/freshports
data02/freshports/dvl-ingress01/ingress                             mountpoint  /jails/dvl-ingress01/var/db/ingress                              received
data02/freshports/dvl-ingress01/ingress/latest_commits              mountpoint  /jails/dvl-ingress01/var/db/ingress/latest_commits               inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/ingress/message-queues              mountpoint  /jails/dvl-ingress01/var/db/ingress/message-queues               inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/ingress/repos                       mountpoint  /jails/dvl-ingress01/var/db/ingress/repos                        inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/ingress/repos/doc                   mountpoint  /jails/dvl-ingress01/var/db/ingress/repos/doc                    inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/ingress/repos/ports                 mountpoint  /jails/dvl-ingress01/var/db/ingress/repos/ports                  inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/ingress/repos/src                   mountpoint  /jails/dvl-ingress01/var/db/ingress/repos/src                    inherited from data02/freshports/dvl-ingress01/ingress
data02/freshports/dvl-ingress01/jails                               mountpoint  /jails/dvl-ingress01/jails                                       received
data02/freshports/dvl-ingress01/jails/freshports                    mountpoint  /jails/dvl-ingress01/jails/freshports                            inherited from data02/freshports/dvl-ingress01/jails
data02/freshports/dvl-ingress01/jails/freshports/ports              mountpoint  /jails/dvl-ingress01/jails/freshports/usr/ports                  received
data02/freshports/dvl-ingress01/modules                             mountpoint  /jails/dvl-ingress01/usr/local/lib/perl5/site_perl/FreshPorts    received
data02/freshports/dvl-ingress01/scripts                             mountpoint  /jails/dvl-ingress01/usr/local/libexec/freshports                received
data02/freshports/dvl-nginx01                                       mountpoint  none                                                             inherited from data02
data02/freshports/dvl-nginx01/www                                   mountpoint  none                                                             inherited from data02
data02/freshports/dvl-nginx01/www/freshports                        mountpoint  /jails/dvl-nginx01/usr/local/www/freshports                      received
data02/freshports/dvl-nginx01/www/freshsource                       mountpoint  /jails/dvl-nginx01/usr/local/www/freshsource                     received
data02/freshports/jailed                                            mountpoint  none                                                             inherited from data02
data02/freshports/jailed/dev-ingress01                              mountpoint  none                                                             inherited from data02
data02/freshports/jailed/dev-nginx01                                mountpoint  none                                                             inherited from data02
data02/freshports/jailed/dev-nginx01/cache                          mountpoint  /var/db/freshports/cache                                         received
data02/freshports/jailed/dev-nginx01/cache/categories               mountpoint  /var/db/freshports/cache/categories                              inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/commits                  mountpoint  /var/db/freshports/cache/commits                                 inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/daily                    mountpoint  /var/db/freshports/cache/daily                                   inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/general                  mountpoint  /var/db/freshports/cache/general                                 inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/news                     mountpoint  /var/db/freshports/cache/news                                    inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/packages                 mountpoint  /var/db/freshports/cache/packages                                inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/pages                    mountpoint  /var/db/freshports/cache/pages                                   inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/ports                    mountpoint  /var/db/freshports/cache/ports                                   inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dev-nginx01/cache/spooling                 mountpoint  /var/db/freshports/cache/spooling                                inherited from data02/freshports/jailed/dev-nginx01/cache
data02/freshports/jailed/dvl-ingress01                              mountpoint  none                                                             received
data02/freshports/jailed/dvl-ingress01/distfiles                    mountpoint  none                                                             inherited from data02/freshports/jailed/dvl-ingress01
data02/freshports/jailed/dvl-nginx01                                mountpoint  none                                                             inherited from data02
data02/freshports/jailed/dvl-nginx01/cache                          mountpoint  /var/db/freshports/cache                                         received
data02/freshports/jailed/dvl-nginx01/cache/categories               mountpoint  /var/db/freshports/cache/categories                              inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/commits                  mountpoint  /var/db/freshports/cache/commits                                 inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/daily                    mountpoint  /var/db/freshports/cache/daily                                   inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/general                  mountpoint  /var/db/freshports/cache/general                                 inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/news                     mountpoint  /var/db/freshports/cache/news                                    inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/packages                 mountpoint  /var/db/freshports/cache/packages                                inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/pages                    mountpoint  /var/db/freshports/cache/pages                                   inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/ports                    mountpoint  /var/db/freshports/cache/ports                                   inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/cache/spooling                 mountpoint  /var/db/freshports/cache/spooling                                inherited from data02/freshports/jailed/dvl-nginx01/cache
data02/freshports/jailed/dvl-nginx01/freshports                     mountpoint  none                                                             inherited from data02
data02/freshports/jailed/stage-ingress01                            mountpoint  none                                                             inherited from data02
data02/freshports/jailed/stage-ingress01/data                       mountpoint  none                                                             inherited from data02
data02/freshports/jailed/stage-nginx01                              mountpoint  none                                                             inherited from data02
data02/freshports/jailed/stage-nginx01/cache                        mountpoint  /var/db/freshports/cache                                         received
data02/freshports/jailed/stage-nginx01/cache/categories             mountpoint  /var/db/freshports/cache/categories                              inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/commits                mountpoint  /var/db/freshports/cache/commits                                 inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/daily                  mountpoint  /var/db/freshports/cache/daily                                   inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/general                mountpoint  /var/db/freshports/cache/general                                 inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/news                   mountpoint  /var/db/freshports/cache/news                                    inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/packages               mountpoint  /var/db/freshports/cache/packages                                inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/pages                  mountpoint  /var/db/freshports/cache/pages                                   inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/ports                  mountpoint  /var/db/freshports/cache/ports                                   inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/stage-nginx01/cache/spooling               mountpoint  /var/db/freshports/cache/spooling                                inherited from data02/freshports/jailed/stage-nginx01/cache
data02/freshports/jailed/test-ingress01                             mountpoint  none                                                             inherited from data02
data02/freshports/jailed/test-ingress01/data                        mountpoint  none                                                             inherited from data02
data02/freshports/jailed/test-nginx01                               mountpoint  none                                                             inherited from data02
data02/freshports/jailed/test-nginx01/cache                         mountpoint  /var/db/freshports/cache                                         received
data02/freshports/jailed/test-nginx01/cache/categories              mountpoint  /var/db/freshports/cache/categories                              inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/commits                 mountpoint  /var/db/freshports/cache/commits                                 inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/daily                   mountpoint  /var/db/freshports/cache/daily                                   inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/general                 mountpoint  /var/db/freshports/cache/general                                 inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/news                    mountpoint  /var/db/freshports/cache/news                                    inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/packages                mountpoint  /var/db/freshports/cache/packages                                inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/pages                   mountpoint  /var/db/freshports/cache/pages                                   inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/ports                   mountpoint  /var/db/freshports/cache/ports                                   inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/jailed/test-nginx01/cache/spooling                mountpoint  /var/db/freshports/cache/spooling                                inherited from data02/freshports/jailed/test-nginx01/cache
data02/freshports/stage-ingress01                                   mountpoint  none                                                             inherited from data02
data02/freshports/stage-ingress01/cache                             mountpoint  /jails/stage-ingress01/var/db/freshports/cache                   received
data02/freshports/stage-ingress01/cache/html                        mountpoint  /jails/stage-ingress01/var/db/freshports/cache/html              inherited from data02/freshports/stage-ingress01/cache
data02/freshports/stage-ingress01/cache/spooling                    mountpoint  /jails/stage-ingress01/var/db/freshports/cache/spooling          inherited from data02/freshports/stage-ingress01/cache
data02/freshports/stage-ingress01/freshports                        mountpoint  none                                                             inherited from data02
data02/freshports/stage-ingress01/freshports/archive                mountpoint  /jails/stage-ingress01/var/db/freshports/message-queues/archive  received
data02/freshports/stage-ingress01/freshports/message-queues         mountpoint  /jails/stage-ingress01/var/db/freshports/message-queues          received
data02/freshports/stage-ingress01/ingress                           mountpoint  /jails/stage-ingress01/var/db/ingress                            received
data02/freshports/stage-ingress01/ingress/latest_commits            mountpoint  /jails/stage-ingress01/var/db/ingress/latest_commits             inherited from data02/freshports/stage-ingress01/ingress
data02/freshports/stage-ingress01/ingress/message-queues            mountpoint  /jails/stage-ingress01/var/db/ingress/message-queues             inherited from data02/freshports/stage-ingress01/ingress
data02/freshports/stage-ingress01/ingress/repos                     mountpoint  /jails/stage-ingress01/var/db/ingress/repos                      inherited from data02/freshports/stage-ingress01/ingress
data02/freshports/stage-ingress01/jails                             mountpoint  /jails/stage-ingress01/jails                                     received
data02/freshports/stage-ingress01/jails/freshports                  mountpoint  /jails/stage-ingress01/jails/freshports                          inherited from data02/freshports/stage-ingress01/jails
data02/freshports/stage-ingress01/ports                             mountpoint  /jails/stage-ingress01/jails/freshports/usr/ports                received
data02/freshports/test-ingress01                                    mountpoint  none                                                             inherited from data02
data02/freshports/test-ingress01/freshports                         mountpoint  /jails/test-ingress01/var/db/freshports                          received
data02/freshports/test-ingress01/freshports/cache                   mountpoint  /jails/test-ingress01/var/db/freshports/cache                    inherited from data02/freshports/test-ingress01/freshports
data02/freshports/test-ingress01/freshports/cache/html              mountpoint  /jails/test-ingress01/var/db/freshports/cache/html               inherited from data02/freshports/test-ingress01/freshports
data02/freshports/test-ingress01/freshports/cache/spooling          mountpoint  /jails/test-ingress01/var/db/freshports/cache/spooling           inherited from data02/freshports/test-ingress01/freshports
data02/freshports/test-ingress01/freshports/message-queues          mountpoint  /jails/test-ingress01/var/db/freshports/message-queues           inherited from data02/freshports/test-ingress01/freshports
data02/freshports/test-ingress01/freshports/message-queues/archive  mountpoint  /jails/test-ingress01/var/db/freshports/message-queues/archive   inherited from data02/freshports/test-ingress01/freshports
data02/freshports/test-ingress01/ingress                            mountpoint  /jails/test-ingress01/var/db/ingress                             received
data02/freshports/test-ingress01/ingress/latest_commits             mountpoint  /jails/test-ingress01/var/db/ingress/latest_commits              inherited from data02/freshports/test-ingress01/ingress
data02/freshports/test-ingress01/ingress/message-queues             mountpoint  /jails/test-ingress01/var/db/ingress/message-queues              inherited from data02/freshports/test-ingress01/ingress
data02/freshports/test-ingress01/ingress/repos                      mountpoint  /jails/test-ingress01/var/db/ingress/repos                       inherited from data02/freshports/test-ingress01/ingress
data02/freshports/test-ingress01/jails                              mountpoint  /jails/test-ingress01/jails                                      received
data02/freshports/test-ingress01/jails/freshports                   mountpoint  /jails/test-ingress01/jails/freshports                           inherited from data02/freshports/test-ingress01/jails
data02/freshports/test-ingress01/jails/freshports/ports             mountpoint  /jails/test-ingress01/jails/freshports/usr/ports                 received
data02/jails                                                        mountpoint  /jails                                                           received
data02/jails/bacula                                                 mountpoint  /jails/bacula                                                    inherited from data02/jails
data02/jails/bacula-sd-02                                           mountpoint  /jails/bacula-sd-02                                              inherited from data02/jails
data02/jails/bacula-sd-03                                           mountpoint  /jails/bacula-sd-03                                              inherited from data02/jails
data02/jails/besser                                                 mountpoint  /jails/besser                                                    inherited from data02/jails
data02/jails/certs                                                  mountpoint  /jails/certs                                                     inherited from data02/jails
data02/jails/certs-rsync                                            mountpoint  /jails/certs-rsync                                               inherited from data02/jails
data02/jails/cliff2                                                 mountpoint  /jails/cliff2                                                    inherited from data02/jails
data02/jails/dev-ingress01                                          mountpoint  /jails/dev-ingress01                                             inherited from data02/jails
data02/jails/dev-nginx01                                            mountpoint  /jails/dev-nginx01                                               inherited from data02/jails
data02/jails/dev-pgeu                                               mountpoint  /jails/dev-pgeu                                                  inherited from data02/jails
data02/jails/dns-hidden-master                                      mountpoint  /jails/dns-hidden-master                                         inherited from data02/jails
data02/jails/dns1                                                   mountpoint  /jails/dns1                                                      inherited from data02/jails
data02/jails/dvl-ingress01                                          mountpoint  /jails/dvl-ingress01                                             inherited from data02/jails
data02/jails/dvl-nginx01                                            mountpoint  /jails/dvl-nginx01                                               inherited from data02/jails
data02/jails/freebsd143                                             mountpoint  /jails/freebsd143                                                inherited from data02/jails
data02/jails/git                                                    mountpoint  /jails/git                                                       inherited from data02/jails
data02/jails/jail_within_jail                                       mountpoint  /jails/jail_within_jail                                          inherited from data02/jails
data02/jails/mqtt01                                                 mountpoint  /jails/mqtt01                                                    inherited from data02/jails
data02/jails/mydev                                                  mountpoint  /jails/mydev                                                     inherited from data02/jails
data02/jails/mysql01                                                mountpoint  /jails/mysql01                                                   inherited from data02/jails
data02/jails/nsnotify                                               mountpoint  /jails/nsnotify                                                  inherited from data02/jails
data02/jails/pg01                                                   mountpoint  /jails/pg01                                                      inherited from data02/jails
data02/jails/pg02                                                   mountpoint  /jails/pg02                                                      inherited from data02/jails
data02/jails/pg03                                                   mountpoint  /jails/pg03                                                      inherited from data02/jails
data02/jails/pkg01                                                  mountpoint  /jails/pkg01                                                     inherited from data02/jails
data02/jails/samdrucker                                             mountpoint  /jails/samdrucker                                                inherited from data02/jails
data02/jails/serpico                                                mountpoint  /jails/serpico                                                   inherited from data02/jails
data02/jails/stage-ingress01                                        mountpoint  /jails/stage-ingress01                                           inherited from data02/jails
data02/jails/stage-nginx01                                          mountpoint  /jails/stage-nginx01                                             inherited from data02/jails
data02/jails/svn                                                    mountpoint  /jails/svn                                                       inherited from data02/jails
data02/jails/talos                                                  mountpoint  /jails/talos                                                     inherited from data02/jails
data02/jails/test-ingress01                                         mountpoint  /jails/test-ingress01                                            inherited from data02/jails
data02/jails/test-nginx01                                           mountpoint  /jails/test-nginx01                                              inherited from data02/jails
data02/jails/unifi01                                                mountpoint  /jails/unifi01                                                   inherited from data02/jails
data02/jails/webserver                                              mountpoint  /jails/webserver                                                 inherited from data02/jails
data02/reserved                                                     mountpoint  none                                                             inherited from data02
data02/vm                                                           mountpoint  /usr/local/vm                                                    received
data02/vm/freebsd-test                                              mountpoint  /usr/local/vm/freebsd-test                                       inherited from data02/vm
data02/vm/hass                                                      mountpoint  /usr/local/vm/hass                                               inherited from data02/vm
data02/vm/home-assistant                                            mountpoint  /usr/local/vm/home-assistant                                     inherited from data02/vm
data02/vm/myguest                                                   mountpoint  /usr/local/vm/myguest                                            inherited from data02/vm

That looks good.

Next, import these zpools so they are present on the next reboot.

root@r730-01:/home/dvl # zpool import data01
root@r730-01:/home/dvl # zpool import data03

What zpool are not in use?

This lists the zpool which can be imported but are not.

root@r730-01:/home/dvl # zpool import 
   pool: data02_old
     id: 14532602998618854058
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	data02_old                           ONLINE
	  mirror-0                           ONLINE
	    gpt/Samsung_990_S7U8NJ0Y716854P  ONLINE
	    gpt/Samsung_990_S7U8NJ0Y716801F  ONLINE

   pool: data04_old
     id: 14613959245391720618
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	data04_old  ONLINE
	  mirror-0  ONLINE
	    da13    ONLINE
	    da12    ONLINE
root@r730-01:/home/dvl # 

All of those drives are destined to be used by the new big zpool.

Before rebooting

Before I restart, I disable these services. They are done now, after I’ve manually stopped them and before reboot.

root@r730-01:/home/dvl # sysrc jail_enable="NO"
jail_enable: YES -> NO

root@r730-01:/home/dvl # sysrc vm_enable="NO"
vm_enable: YES -> NO

After I reboot the host and know that it looks good, I’ll enable those services, start them manually, and review. If all good, one more restart.

Here is where I tell you: shutdown -r now is not the same as reboot

reboot does not run your rc scripts.

Although I say reboot and restart, that is always accomplished via shutdown -r now

root@r730-01:/home/dvl # shutdown -r now
Shutdown NOW!
shutdown: [pid 52693]
root@r730-01:/home/dvl #                                                                                
*** FINAL System shutdown message from dvl@r730-01.int.unixathome.org ***    

System going down IMMEDIATELY                                                  

                                                                               

System shutdown time has arrived
Connection to r730-01.int.unixathome.org closed by remote host.
Connection to r730-01.int.unixathome.org closed.

After the reboot

Here we go, this looks good:

[17:29 r730-01 dvl ~] % uptime
 5:29PM  up 34 secs, 1 user, load averages: 0.33, 0.09, 0.03
[17:29 r730-01 dvl ~] % zpool list
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data01  5.81T  5.10T   731G        -         -     3%    87%  1.00x    ONLINE  -
data02   928G   646G   282G        -         -     0%    69%  1.00x    ONLINE  -
data03  7.25T  1.29T  5.96T        -         -    33%    17%  1.00x    ONLINE  -
zroot    107G  30.2G  76.8G        -         -    15%    28%  1.00x    ONLINE  -
[17:29 r730-01 dvl ~] % 

Let’s start bhyve first:

[17:31 r730-01 dvl ~] % sudo sysrc vm_enable="YES"
vm_enable: NO -> YES
[17:31 r730-01 dvl ~] % sudo service vm start
[17:31 r730-01 dvl ~] % 

Yes, that looks good. On to the jails.

[17:31 r730-01 dvl ~] % sudo sysrc jail_enable="YES"
jail_enable: NO -> YES
[17:42 r730-01 dvl ~] % sudo service jail start     
Starting jails: dns1 cliff2 mysql01 pg01 pg02 pg03 pkg01 dev-ingress01 dev-nginx01 dvl-ingress01 dvl-nginx01 test-ingress01 test-nginx01 stage-ingress01 stage-nginx01 bacula besser certs-rsync certs git svn webserver mqtt01 bacula-sd-02 talos samdrucker bacula-sd-03 mydev jail_within_jail serpico dns-hidden-master nsnotify unifi01.
[17:43 r730-01 dvl ~] % 

And now I want little while for monitoring to catch up…

It’s now ten minutes later, and the only Nagios alerts are time related:

FILE_AGE WARNING: /var/db/ingress/latest_commits/latest_commits.txt is 6451 seconds old and 3662 bytes

Well, yeah, the host wasn’t up and running to do that…

I declare good-to-go.

Oh wait, I need to do a full proper reboot

sudo shutdown -r now

Yep, that went well. Now it’s time to create that new zpool

Top

Adding gmirror swap to FreeBSD 14.3 – r730-01

Post by Dan Langille via Dan Langille's Other Diary »

I recently migrated the zroot on r730-01 to a new pair of devices. I did not then configure swap for the swap partitions I set up. I’m going to do that now.

I’m hoping this post from 8 years ago helps.

In this post:

  • FreeBSD 14.3

The first few sections include what’s in the system, and detects and fixes a few issues. Look for The real work near the bottom of the page.

What I have now

This is what I have now.

The zpool:

[17:31 r730-01 dvl ~] % zpool status zroot
  pool: zroot
 state: ONLINE
  scan: scrub repaired 0B in 00:01:13 with 0 errors on Thu Dec  4 04:28:41 2025
config:

	NAME                               STATE     READ WRITE CKSUM
	zroot                              ONLINE       0     0     0
	  mirror-0                         ONLINE       0     0     0
	    gpt/zfs0_20170718AA0000185556  ONLINE       0     0     0
	    gpt/zfs1_20170719AA1178164201  ONLINE       0     0     0

errors: No known data errors

What is the geom name for that device? I need that to run gpart on the right devices. I could just run gpart and pick, but I’m trying to do this the “proper” way to avoid mistakes.

Here is what I do: glabel list | less – then I search for gpt/zfs0_20170718AA0000185556 (found under mirror-0 above.

I find this:

Geom name: ada0p3
Providers:
1. Name: gpt/zfs0_20170718AA0000185556
   Mediasize: 115234181120 (107G)

and this:

Geom name: ada1p3
Providers:
1. Name: gpt/zfs1_20170719AA1178164201
   Mediasize: 115234181120 (107G)

Let’s look at those disks:

[17:41 r730-01 dvl ~] % gpart show ada0 ada1
=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

There, those are them. I have better names set up. Look here:

[17:41 r730-01 dvl ~] % gpart show -l ada0 ada1
=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi0_20170718AA0000185556  (200M)
     411648   16777216     2  swap0  (8.0G)
   17188864  225066760     3  zfs0_20170718AA0000185556  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi1_20170719AA1178164201  (200M)
     411648   16777216     2  swap1  (8.0G)
   17188864  225066760     3  zfs1_20170719AA1178164201  (107G)

I want these devices:

[17:41 r730-01 dvl ~] % ls -l /dev/gpt/swap*
crw-r-----  1 root operator 0xd5 2025.11.29 17:06 /dev/gpt/swap0
crw-r-----  1 root operator 0xdf 2025.11.29 17:06 /dev/gpt/swap1

They are 8G each.

Let’s go.

Creating the gmirror

Unlike my previous post, I already have a gmirror, loaded:

[17:45 r730-01 dvl ~] % grep mirror /boot/loader.conf
geom_mirror_load="YES"

[17:42 r730-01 dvl ~] % gmirror list -a
[17:43 r730-01 dvl ~] % 

Adding in the first half of the mirror – and getting an unexpected result

This statement creates a geom named swap, and adds one device to it:

[17:47 r730-01 dvl ~] % sudo gmirror label -v -b round-robin swap /dev/gpt/swap0  
Metadata value stored on /dev/gpt/swap0.
Done.
[17:47 r730-01 dvl ~] % gmirror list
Geom name: swap
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 2519922271
Type: AUTOMATIC
Providers:
1. Name: mirror/swap
   Mediasize: 2147483136 (2.0G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: nda4p2
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 1048576
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: (null)
   GenID: 0
   SyncID: 1
   ID: 688737778

[17:47 r730-01 dvl ~] % 

nda4p2? 2G? What?

Well, I have more than one device with the same label.

[17:50 r730-01 dvl ~] % gpart show -l | grep swap0
       2048    4194304     2  swap0  (2.0G)
     411648   16777216     2  swap0  (8.0G)

Deleting the extra swap labels

What do I do when there is more than one device with the same label? Remove one label.

This is the problem device. There may be more.

[17:50 r730-01 dvl ~] % gpart show -l | less      
=>       40  488397088  nda4  GPT  (233G)
         40       1024     1  gptboot0  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  swap0  (2.0G)
    4196352  484200448     3  zfs0  (231G)
  488396800        328        - free -  (164K)

This removes that label:

[17:53 r730-01 dvl ~] % sudo gpart modify -i 2 -l '' nda4
nda4p2 modified
[17:54 r730-01 dvl ~] % gpart show nda4
=>       40  488397088  nda4  GPT  (233G)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  freebsd-swap  (2.0G)
    4196352  484200448     3  freebsd-zfs  (231G)
  488396800        328        - free -  (164K)

[17:54 r730-01 dvl ~] % gpart show -l nda4
=>       40  488397088  nda4  GPT  (233G)
         40       1024     1  gptboot0  (512K)
       1064        984        - free -  (492K)
       2048    4194304     2  (null)  (2.0G)
    4196352  484200448     3  zfs0  (231G)
  488396800        328        - free -  (164K)

[17:54 r730-01 dvl ~] % 

Let’s check for swap1 while here…

Reviewing the output of gpart show -l | less found only one swap1 and it was the 8G one on the right device.

Undoing the work

[17:57 r730-01 dvl ~] % sudo gmirror remove swap /dev/gpt/swap0
gmirror: No such provider: /dev/gpt/swap0.

Well, yes, we deleted that. However, based on other tests, you can’t remove by that type of name. You have to do what I have below:

[17:58 r730-01 dvl ~] % sudo gmirror remove swap gpt/swap0     
gmirror: No such provider: gpt/swap0.
[17:58 r730-01 dvl ~] % sudo gmirror remove swap nda4p2   
gmirror: swap: Can't remove the last ACTIVE component nda4p2.
[17:58 r730-01 dvl ~] % sudo gmirror destroy swap      
[17:59 r730-01 dvl ~] % sudo gmirror list        
[17:59 r730-01 dvl ~] % 

Starting fresh. Let’s do this again

Recreate the gmirror:

[17:59 r730-01 dvl ~] % sudo gmirror label -v -b round-robin swap /dev/gpt/swap0
gmirror: Can't get informations about /dev/gpt/swap0: No such file or directory.
[18:01 r730-01 dvl ~] % gpart show ada0 ada1                                    
=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi  (200M)
     411648   16777216     2  freebsd-swap  (8.0G)
   17188864  225066760     3  freebsd-zfs  (107G)

[18:01 r730-01 dvl ~] % gpart show -l ada0 ada1
=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi0_20170718AA0000185556  (200M)
     411648   16777216     2  swap0  (8.0G)
   17188864  225066760     3  zfs0_20170718AA0000185556  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1.0M)
       2048     409600     1  efi1_20170719AA1178164201  (200M)
     411648   16777216     2  swap1  (8.0G)
   17188864  225066760     3  zfs1_20170719AA1178164201  (107G)

[18:01 r730-01 dvl ~] % 

Oh. It seems you can have only one /dev/gpt/swap0.

Let’s try this to get that device recreated:

[18:01 r730-01 dvl ~] % sudo gpart modify -i 2 -l swap0 ada0                    
ada0p2 modified
[18:03 r730-01 dvl ~] % ls -l /dev/gpt/swap0
crw-r-----  1 root operator 0x216 2025.12.04 18:03 /dev/gpt/swap0
[18:03 r730-01 dvl ~] % 

With that command I set the label to the same value. That created the device I needed.

The real work

This one works.

[18:05 r730-01 dvl ~] % sudo gmirror label -v -b round-robin swap /dev/gpt/swap0
Metadata value stored on /dev/gpt/swap0.
Done.

[18:05 r730-01 dvl ~] % gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ada0p2 (ACTIVE)

[18:05 r730-01 dvl ~] % gmirror list                                            
Geom name: swap
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 436380494
Type: AUTOMATIC
Providers:
1. Name: mirror/swap
   Mediasize: 8589934080 (8.0G)
   Sectorsize: 512
   Mode: r0w0e0
Consumers:
1. Name: ada0p2
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 210763776
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: (null)
   GenID: 0
   SyncID: 1
   ID: 1019612839

[18:05 r730-01 dvl ~] % 

Next, I add in the second device:

[18:05 r730-01 dvl ~] % sudo gmirror insert swap /dev/gpt/swap1
[18:06 r730-01 dvl ~] % gmirror status
       Name    Status  Components
mirror/swap  DEGRADED  ada0p2 (ACTIVE)
                       ada1p2 (SYNCHRONIZING, 14%)
[18:06 r730-01 dvl ~] % gmirror list
Geom name: swap
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 436380494
Type: AUTOMATIC
Providers:
1. Name: mirror/swap
   Mediasize: 8589934080 (8.0G)
   Sectorsize: 512
   Mode: r1w0e0
Consumers:
1. Name: ada0p2
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 210763776
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: (null)
   GenID: 0
   SyncID: 1
   ID: 1019612839
2. Name: ada1p2
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 210763776
   Mode: r1w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   GenID: 0
   SyncID: 1
   BytesSynced: 3457155072
   Synchronized: 40%
   ID: 2160596712

[18:06 r730-01 dvl ~] % 

That’s much better.

By the time I’d pasted all that into here, the gmirror was stable:

[18:06 r730-01 dvl ~] % gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
                       ada1p2 (ACTIVE)

Hold on a minute! 2025-12-08

I’ve created a gmirror, using the swap partitions, but FreeBSD doesn’t know it is to be used for swap.

Based on my recent Mastodon post,

swapinfo

knows nothing:

[23:31 r730-01 dvl ~] % swapinfo
Device          1K-blocks     Used    Avail Capacity

I do have an entry in /etc/fstab, but what it refers to does not exist:

[0:06 r730-01 dvl ~] % cat /etc/fstab    
# Device		Mountpoint	FStype	Options		Dump	Pass#
/dev/mirror/swap.eli		none	swap	sw		0	0

I also want to enable encryption on that swap partition (that is what the fstab entry is pointing to ). Let’s be extremist and follow the docs.

Let’s create existing swap partitions.

[23:39 r730-01 dvl ~] % gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
                       ada1p2 (ACTIVE)

Based on that, this is how I overwrite that:

[23:59 r730-01 dvl ~] % sudo dd if=/dev/random bs=1m of=/dev/mirror/swap
load: 3.72  cmd: dd 93474 [physwr] 44.83r 0.03u 23.91s 55% 3440k
6820+0 records in
6820+0 records out
7151288320 bytes transferred in 44.831813 secs (159513698 bytes/sec)
dd: /dev/mirror/swap: short write on character device
dd: /dev/mirror/swap: end of device
8192+0 records in
8191+1 records out
8589934080 bytes transferred in 53.844142 secs (159533306 bytes/sec)

I pressed ^T there, and it told me it had written about 7.15GB.

Starting swap

Starting swap was… easy?

[0:09 r730-01 dvl ~] % ls -l /dev/mirror  
total 0
crw-r-----  1 root operator 0x653 2025.12.09 00:00 swap

[0:11 r730-01 dvl ~] % sudo service swap start

[0:11 r730-01 dvl ~] % ls -l /dev/mirror
total 0
crw-r-----  1 root operator 0x653 2025.12.09 00:00 swap
crw-r-----  1 root operator 0x61c 2025.12.09 00:11 swap.eli

[0:11 r730-01 dvl ~] % swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/mirror/swap.eli   8388604        0  8388604     0%
[0:11 r730-01 dvl ~] % 

So that’s it now, swap is working. Finally.

Not shown here, is the addition of this value to this file:

[0:11 r730-01 dvl ~] % grep mirror /boot/loader.conf
geom_mirror_load="YES"

Hope that helps.

Top

Linux kernel version numbers

Post by Greg Kroah-Hartman via Linux Kernel Monkey Log »

Despite having a stable release model and cadence since December 2003, Linux kernel version numbers seem to baffle and confuse those that run across them, causing numerous groups to mistakenly make versioning statements that are flat out false. So let’s go into how this all works in detail.

Top

Google Sues to Disrupt Chinese SMS Phishing Triad

Post by Brian Krebs via Krebs on Security »

Google is suing more than two dozen unnamed individuals allegedly involved in peddling a popular China-based mobile phishing service that helps scammers impersonate hundreds of trusted brands, blast out text message lures, and convert phished payment card data into mobile wallets from Apple and Google.

In a lawsuit filed in the Southern District of New York on November 12, Google sued to unmask and disrupt 25 “John Doe” defendants allegedly linked to the sale of Lighthouse, a sophisticated phishing kit that makes it simple for even novices to steal payment card data from mobile users. Google said Lighthouse has harmed more than a million victims across 120 countries.

A component of the Chinese phishing kit Lighthouse made to target customers of The Toll Roads, which refers to several state routes through Orange County, Calif.

Lighthouse is one of several prolific phishing-as-a-service operations known as the “Smishing Triad,” and collectively they are responsible for sending millions of text messages that spoof the U.S. Postal Service to supposedly collect some outstanding delivery fee, or that pretend to be a local toll road operator warning of a delinquent toll fee. More recently, Lighthouse has been used to spoof e-commerce websites, financial institutions and brokerage firms.

Regardless of the text message lure or brand used, the basic scam remains the same: After the visitor enters their payment information, the phishing site will automatically attempt to enroll the card as a mobile wallet from Apple or Google. The phishing site then tells the visitor that their bank is going to verify the transaction by sending a one-time code that needs to be entered into the payment page before the transaction can be completed.

If the recipient provides that one-time code, the scammers can link the victim’s card data to a mobile wallet on a device that they control. Researchers say the fraudsters usually load several stolen wallets onto each mobile device, and wait 7-10 days after that enrollment before selling the phones or using them for fraud.

Google called the scale of the Lighthouse phishing attacks “staggering.” A May 2025 report from Silent Push found the domains used by the Smishing Triad are rotated frequently, with approximately 25,000 phishing domains active during any 8-day period.

Google’s lawsuit alleges the purveyors of Lighthouse violated the company’s trademarks by including Google’s logos on countless phishing websites. The complaint says Lighthouse offers over 600 templates for phishing websites of more than 400 entities, and that Google’s logos were featured on at least a quarter of those templates.

Google is also pursuing Lighthouse under the Racketeer Influenced and Corrupt Organizations (RICO) Act, saying the Lighthouse phishing enterprise encompasses several connected threat actor groups that work together to design and implement complex criminal schemes targeting the general public.

According to Google, those threat actor teams include a “developer group” that supplies the phishing software and templates; a “data broker group” that provides a list of targets; a “spammer group” that provides the tools to send fraudulent text messages in volume; a “theft group,” in charge of monetizing the phished information; and an “administrative group,” which runs their Telegram support channels and discussion groups designed to facilitate collaboration and recruit new members.

“While different members of the Enterprise may play different roles in the Schemes, they all collaborate to execute phishing attacks that rely on the Lighthouse software,” Google’s complaint alleges. “None of the Enterprise’s Schemes can generate revenue without collaboration and cooperation among the members of the Enterprise. All of the threat actor groups are connected to one another through historical and current business ties, including through their use of Lighthouse and the online community supporting its use, which exists on both YouTube and Telegram channels.”

Silent Push’s May report observed that the Smishing Triad boasts it has “300+ front desk staff worldwide” involved in Lighthouse, staff that is mainly used to support various aspects of the group’s fraud and cash-out schemes.

An image shared by an SMS phishing group shows a panel of mobile phones responsible for mass-sending phishing messages. These panels require a live operator because the one-time codes being shared by phishing victims must be used quickly as they generally expire within a few minutes.

Google alleges that in addition to blasting out text messages spoofing known brands, Lighthouse makes it easy for customers to mass-create fake e-commerce websites that are advertised using Google Ads accounts (and paid for with stolen credit cards). These phony merchants collect payment card information at checkout, and then prompt the customer to expect and share a one-time code sent from their financial institution.

Once again, that one-time code is being sent by the bank because the fake e-commerce site has just attempted to enroll the victim’s payment card data in a mobile wallet. By the time a victim understands they will likely never receive the item they just purchased from the fake e-commerce shop, the scammers have already run through hundreds of dollars in fraudulent charges, often at high-end electronics stores or jewelers.

Ford Merrill works in security research at SecAlliance, a CSIS Security Group company, and he’s been tracking Chinese SMS phishing groups for several years. Merrill said many Lighthouse customers are now using the phishing kit to erect fake e-commerce websites that are advertised on Google and Meta platforms.

“You find this shop by searching for a particular product online or whatever, and you think you’re getting a good deal,” Merrill said. “But of course you never receive the product, and they will phish that one-time code at checkout.”

Merrill said some of the phishing templates include payment buttons for services like PayPal, and that victims who choose to pay through PayPal can also see their PayPal accounts hijacked.

A fake e-commerce site from the Smishing Triad spoofing PayPal on a mobile device.

“The main advantage of the fake e-commerce site is that it doesn’t require them to send out message lures,” Merrill said, noting that the fake vendor sites have more staying power than traditional phishing sites because it takes far longer for them to be flagged for fraud.

Merrill said Google’s legal action may temporarily disrupt the Lighthouse operators, and could make it easier for U.S. federal authorities to bring criminal charges against the group. But he said the Chinese mobile phishing market is so lucrative right now that it’s difficult to imagine a popular phishing service voluntarily turning out the lights.

Merrill said Google’s lawsuit also can help lay the groundwork for future disruptive actions against Lighthouse and other phishing-as-a-service entities that are operating almost entirely on Chinese networks. According to Silent Push, a majority of the phishing sites created with these kits are sitting at two Chinese hosting companies: Tencent (AS132203) and Alibaba (AS45102).

“Once Google has a default judgment against the Lighthouse guys in court, theoretically they could use that to go to Alibaba and Tencent and say, ‘These guys have been found guilty, here are their domains and IP addresses, we want you to shut these down or we’ll include you in the case.'”

If Google can bring that kind of legal pressure consistently over time, Merrill said, they might succeed in increasing costs for the phishers and more frequently disrupting their operations.

“If you take all of these Chinese phishing kit developers, I have to believe it’s tens of thousands of Chinese-speaking people involved,” he said. “The Lighthouse guys will probably burn down their Telegram channels and disappear for a while. They might call it something else or redevelop their service entirely. But I don’t believe for a minute they’re going to close up shop and leave forever.”

Top

Valuable News – 2025/12/08

Post by Vermaden via 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗 »

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX/BSD/Linux systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here.

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

The Usual Suspects section at the end is permanent and have links to other sites with interesting UNIX/BSD/Linux news.

Past releases are available at the dedicated NEWS page.

UNIX

FreeBSD 15.0-RELEASE Now Available.
https://lists.freebsd.org/archives/freebsd-announce/2025-December/000213.html

The gurk-rs is Signal Messenger Client for Terminal.
https://github.com/boxdot/gurk-rs/

FreeBSD Git Weekly for 2025-11-24 – 2025-11-30.
https://freebsd-git-weekly.tarsnap.net/2025-11-24.html

FreeBSD 15.0 Now Officially Available with Many Software Updates and Reproducible Builds.
https://phoronix.com/news/FreeBSD-15.0-RELEASE

The hbsd-update(8) Signing Material Rotation on HardenedBSD.
https://groups.google.com/a/hardenedbsd.org/g/users/c/l-UivSvXK5c/m/6hX7SknqBAAJ

FreeBSD 15.0-RELEASE Now Available – Revamps Package Management and Drops 32bit Hardware.
https://howtogeek.com/freebsd-15-now-available-revamps-package-management-and-drops-32-bit-hardware/

Plex: Another Good Project Bites the Dust.
https://vulcanridr.mataroa.blog/blog/plex-another-good-project-bites-the-dust/

Next Steps in Post Plex World.
https://vulcanridr.mataroa.blog/blog/next-steps-in-a-post-plex-world/

FreeBSD 15.0 Released: Major Upgrades to OpenZFS/OpenSSL and Expanded Hardware Support.
https://pbxscience.com/freebsd-15-0-released-major-upgrades-to-openzfs-openssl-and-expanded-hardware-support/

FreeBSD 15.0 is Here – Whats New and What to Expect.
https://maxiujun.com/freebsd-15-0-is-here-whats-new-and-what-to-expect/

FreeBSD 15.0 Released.
https://opensourcefeed.org/freebsd-15-0-released/

FreeBSD 15.0-RELEASE is Out.
https://youtube.com/watch?v=xR0zjPtix50

Bareos 25 Released – New Plugins and Features.
https://bareos.com/bareos25/

Intel Open Source Graphics Driver Delivered Significant Improvements in 2025.
https://phoronix.com/review/intel-b580-opengl-vulkan-eoy2025

How Many VDEVs is Too Many: ZFS VDEV Scaling Guide.
https://klarasystems.com/articles/how-many-vdevs-is-too-many-zfs-vdev-scaling/

How to Configure X11 in Simple Way on FreeBSD.
https://eugene-andrienko.com/it/2025/07/24/x11-configuration-simple.html

FreeBSD 15: Great Strides Thanks to Industry Support.
https://heise.de/en/news/FreeBSD-15-Great-strides-thanks-to-industry-support-11101176.html

ReactOS Lands Improvements for Its USB Stack.
https://phoronix.com/news/Better-USB-For-ReactOS

FreeBSD 15: Why You will Want It.
https://freebsdfoundation.org/blog/freebsd-15-why-youll-want-it/

FreeBSD 15.0 Released – Fresh Chapter for BSD Base System.
https://thedistrowriteproject.blogspot.com/2025/12/FreeBSD-15-0-Released-A-Fresh-Chapter-for-the-BSD-Base-System.html

OPNsense: New Unbound DNS Blocklists.
https://youtube.com/watch?v=lPN9V28xW-4

What We Built: Top ZFS Capabilities Delivered by Klara in 2025.
https://klarasystems.com/articles/zfs-features-by-klara-in-2025/

Couple of GUI Apps for Your FreeBSD Desktop.
https://youtube.com/watch?v=uLhwUPvqykk

FreeBSD Officially Supported in OCI 1.3 Runtime Specification.
https://freebsdfoundation.org/blog/freebsd-officially-supported-in-oci-runtime-specification-v1-3/

FreeBSD 15.0 Benchmarks versus FreeBSD 14.3 on AMD EPYC.
https://phoronix.com/review/freebsd-15-amd-epyc

Wlasny Serwer FreeBSD – Czesc 2 – Serwer WWW Nginx. [Polish]
https://linuxiarze.pl/wlasny-serwer-freebsd-cz-2-serwer-www-nginx/

Why I am Trading My Linux for FreeBSD – Return to Simplicity.
https://artemiesolomon.eu/blog/freebsd-experience/

BSD Now 640: Cleaning up Hammer.
https://www.bsdnow.tv/640

FreeBSD 15 Trims Legacy Fat and Revamps How OS is Built.
https://theregister.com/2025/12/05/freebsd_15/

Yes Book of PF (4th Edition) is Coming Soon.
https://nxdomain.no/~peter/yes_the_book_of_pf_4th_ed_is_coming.html

How to Check FreeBSD Version: Beginners Guide.
https://ostechnix.com/check-freebsd-version/

GitHub to Codeberg: My Experience.
https://eldred.fr/blog/forge-migration/

Jail/Zones 2025/12/02 Production User Call.
https://youtube.com/watch?v=hTwdy8ihawA

Self Hosting Your Mastodon Media with SeaweedFS.
https://it-notes.dragas.net/2025/11/06/self-hosting-your-mastodon-media-with-seaweedfs/

FreeBSD 15.0 Released with PKGBASE Support and Enhanced Security.
https://ostechnix.com/freebsd-15-released/

PuffyGuard is Low Effort Way to Automatically Deploy OpenBSD Machines Running WireGuard.
https://github.com/leahneukirchen/puffyguard

Whats New in Oracle Solaris 11.4 SRU 87.
https://blogs.oracle.com/solaris/whats-new-in-oracle-solaris-11-4-sru-87

Upgrade Guide: Moving from FreeBSD 14.x to 15.0-RELEASE.
https://maxiujun.com/upgrade-guide-moving-from-freebsd-14-x-to-15-0-release/

Ode To OpenBSD.
https://tech-idea.systems/blog-posts/ode-to-openbsd.html

FreeBSD 15.0 Goes Official.
https://officialaptivi.wordpress.com/2025/12/06/freebsd-15-0-goes-official/

How to Install FreeBSD 15 Step by Step.
https://ostechnix.com/install-freebsd/

FreeBSD Networking on Laptop is Unpleasant.
https://blog.feld.me/posts/2025/12/freebsd-networking-laptop-unpleasant/

Perl Decline Was Cultural.
https://beatworm.co.uk/blog/computers/perls-decline-was-cultural-not-technical

Linux and BSD: Two Sides of Same Coin.
https://youtube.com/watch?v=RcY7NxpfWHY

I Do Not Care for Gnome.
https://woltman.com/gnome-bad/

Migrating burningboard.net Mastodon Instance to Multi Jail FreeBSD Setup.
https://blog.hofstede.it/migrating-burningboardnet-mastodon-instance-to-a-multi-jail-freebsd-setup/

New FreeBSD 15 Retires 32bit Ports and Modernizes Builds.
https://tech.slashdot.org/story/25/12/07/0627214/new-freebsd-15-retires-32-bit-ports-and-modernizes-builds

f3s: Kubernetes with FreeBSD – Part 8 – Observability.
https://foo.zone/gemfeed/2025-12-07-f3s-kubernetes-with-freebsd-part-8.html

From Azure Functions to FreeBSD.
https://blogsystem5.substack.com/p/from-azure-functions-to-freebsd

Hardware

ThinkPad X210Ai Completed Laptop with Intel Ultra 7 165h.
https://tpart.net/product/x210ai-completed-laptop-ultra-7-165h/

Hundreds of Porsche Owners in Russia Unable to Start Cars After System Failure.
https://themoscowtimes.com/2025/12/02/hundreds-of-porsche-owners-in-russia-unable-to-start-cars-after-system-failure-a91302

After Nearly 30 years Crucial Will Stop Selling RAM to Consumers.
https://arstechnica.com/gadgets/2025/12/after-nearly-30-years-crucial-will-stop-selling-ram-to-consumers/

AI Data Center Markets are So Big that Micron is Exiting its Crucial Consumer Business.
https://servethehome.com/ai-data-center-markets-are-so-big-that-micron-is-exiting-its-crucial-consumer-business/

IndyCar vs Formula 1: Key Differences in Cars/Engines/Technology.
https://www.openwheelworld.net/en/indycar101/76/IndyCar_vs_Formula_1_cars

Oops. I Accidentally Built Steam Machine.
https://ounapuu.ee/posts/2025/12/01/steam-machine/

1994 McLaren F1 Signed by Schumacher and Hamilton.
https://rmsothebys.com/auctions/ad25/lots/r0004-1994-mclaren-f1/

2025 Gordon Murray Automotive T.50.
https://rmsothebys.com/auctions/ad25/lots/r0003-2025-gordon-murray-automotive-t50/

When Less is So Much More.
https://rmsothebys.com/all-stories-videos/when-less-is-so-much-more/

Worst UK Police Stop and Search I Have Ever Seen.
https://youtube.com/watch?v=UA8RtP3ZlPo

Judge Gives Enoch Burke Effective Life Sentence in Prison.
https://youtube.com/watch?v=pFcNQSg6XRo

Sony Alpha a7 V Review.
https://dpreview.com/reviews/sony-a7-v-review

Life

AI Title Generator.
https://alexnuttinck.dev/posts/7-ai-title-generator/

Man Asks $75 Million for lambo.com Domain – Courts Gives It to Lamborghini for Free.
https://roadandtrack.com/news/a69634055/75-million-dollar-lambo-website-domain-asking-price-lamborghini-court/

Mortal Kombat/Tekken/Man in the High Castle Actor Cary Hiroyuki Tagawa Has Passed Away.
https://timeextension.com/news/2025/12/mortal-kombat-tekken-and-the-man-in-the-high-castle-actor-cary-hiroyuki-tagawa-has-passed-away

Super Rare Hybrid Blood Type Discovered in Just 3 People.
https://sciencealert.com/super-rare-hybrid-blood-type-discovered-in-just-3-people

Divorce Plunged in Kentucky. Equal Custody for Fathers is Big Reason Why.
https://msn.com/en-us/news/us/the-equal-custody-experiment/ar-AA1LXz6l

Female Psychology and End of Empires.
https://youtube.com/watch?v=sRY_1JRRcNU

Other

LibreWolf 145.0.2 Release.
https://codeberg.org/librewolf/bsys6/releases/tag/145.0.2-2

Rootless Pings in Rust.
https://bou.ke/blog/rust-ping/

URL in C.
https://susam.net/url-in-c.html

Coding Trance Music from Scratch (Again).
https://youtube.com/watch?v=iu5rnQkfO6M

YouTube is Now Using AI to Silently Alter Videos without Permission.
https://social.growyourown.services/@FediTips/115668457530054406

I Cracked $200 Software Protection in Day with xcopy Command.
https://ud2.rip/blog/enigma-protector/

Usual Suspects

BSD Weekly.
https://bsdweekly.com/

DiscoverBSD.
https://discoverbsd.com/

BSDSec.
https://bsdsec.net/

DragonFly BSD Digest.
https://dragonflydigest.com/

FreeBSD Patch Level Table.
https://bokut.in/freebsd-patch-level-table/

FreeBSD End of Life Date.
https://endoflife.date/freebsd

Phoronix BSD News Archives.
https://phoronix.com/linux/BSD

OpenBSD Journal.
https://undeadly.org/

Call for Testing.
https://callfortesting.org/

Call for Testing – Production Users Call.
https://youtube.com/@callfortesting/videos

BSD Now Weekly Podcast.
https://www.bsdnow.tv/

Nixers Newsletter.
https://newsletter.nixers.net/entries.php

BSD Cafe Journal.
https://journal.bsd.cafe/

DragonFly BSD Digest – Lazy Reading – In Other BSDs.
https://dragonflydigest.com

BSDTV.
https://bsky.app/profile/bsdtv.bsky.social

FreeBSD Git Weekly.
https://freebsd-git-weekly.tarsnap.net/

FreeBSD Meetings.
https://youtube.com/@freebsdmeetings

BSDJedi.
https://youtube.com/@BSDJedi/videos

RoboNuggie.
https://youtube.com/@RoboNuggie/videos

GaryHTech.
https://youtube.com/@GaryHTech/videos

Sheridan Computers.
https://youtube.com/@sheridans/videos

82MHz.
https://82mhz.net/

EOF
Top

Linux CVEs, more than you ever wanted to know

Post by Greg Kroah-Hartman via Linux Kernel Monkey Log »

It’s been almost 2 full years since Linux became a CNA (Certificate Numbering Authority) which meant that we (i.e. the kernel.org community) are now responsible for issuing all CVEs for the Linux kernel. During this time, we’ve become one of the largest creators of CVEs by quantity, going from nothing to number 3 in 2024 to number 1 in 2025. Naturally, this has caused some questions about how we are both doing all of this work, and how people can keep track of it.

Top

Blinking the SGI Indy Power Light

Post by Adrian Chadd via Adrian Chadd's Ramblings »

I figured out what broke in netbsd to make the R5000SC I have here poweroff during startup.

In any case, the hack for blinking the front panel LED, inspired from their BLINKY driver, shouldn't be lost when i clean up my tree.

 

So, I give you all:

+void
+blink_led(void)
+{
+       uint32_t reg;
+       uint32_t i;
+
+       reg = *(volatile uint32_t *)MIPS_PHYS_TO_KSEG1(0x1fbd9870) & 0xff;
+       reg = reg ^ 0x10; /* toggle? */
+       *(volatile uint32_t *)MIPS_PHYS_TO_KSEG1(0x1fbd9870) = reg;
+       asm(".set mips3; sync");
+
+       for (i = 0; i < 20 * 1000 * 1000; i++)
+               asm(".set mips3; nop ; nop");
+}

It's terrible but it works!

Also, amusingly, the ARC BIOS console IO was available at this point but nothing was being printed early enough for it to be seen! Debugging was quite a bit faster once I realised I was getting far enough for the ARC BIOS entrypoint to be initialised so I could printf().

Top

SMS Phishers Pivot to Points, Taxes, Fake Retailers

Post by Brian Krebs via Krebs on Security »

China-based phishing groups blamed for non-stop scam SMS messages about a supposed wayward package or unpaid toll fee are promoting a new offering, just in time for the holiday shopping season: Phishing kits for mass-creating fake but convincing e-commerce websites that convert customer payment card data into mobile wallets from Apple and Google. Experts say these same phishing groups also are now using SMS lures that promise unclaimed tax refunds and mobile rewards points.

Over the past week, thousands of domain names were registered for scam websites that purport to offer T-Mobile customers the opportunity to claim a large number of rewards points. The phishing domains are being promoted by scam messages sent via Apple’s iMessage service or the functionally equivalent RCS messaging service built into Google phones.

An instant message spoofing T-Mobile says the recipient is eligible to claim thousands of rewards points.

The website scanning service urlscan.io shows thousands of these phishing domains have been deployed in just the past few days alone. The phishing websites will only load if the recipient visits with a mobile device, and they ask for the visitor’s name, address, phone number and payment card data to claim the points.

A phishing website registered this week that spoofs T-Mobile.

If card data is submitted, the site will then prompt the user to share a one-time code sent via SMS by their financial institution. In reality, the bank is sending the code because the fraudsters have just attempted to enroll the victim’s phished card details in a mobile wallet from Apple or Google. If the victim also provides that one-time code, the phishers can then link the victim’s card to a mobile device that they physically control.

Pivoting off these T-Mobile phishing domains in urlscan.io reveals a similar scam targeting AT&T customers:

An SMS phishing or “smishing” website targeting AT&T users.

Ford Merrill works in security research at SecAlliance, a CSIS Security Group company. Merrill said multiple China-based cybercriminal groups that sell phishing-as-a-service platforms have been using the mobile points lure for some time, but the scam has only recently been pointed at consumers in the United States.

“These points redemption schemes have not been very popular in the U.S., but have been in other geographies like EU and Asia for a while now,” Merrill said.

A review of other domains flagged by urlscan.io as tied to this Chinese SMS phishing syndicate shows they are also spoofing U.S. state tax authorities, telling recipients they have an unclaimed tax refund. Again, the goal is to phish the user’s payment card information and one-time code.

A text message that spoofs the District of Columbia’s Office of Tax and Revenue.

CAVEAT EMPTOR

Many SMS phishing or “smishing” domains are quickly flagged by browser makers as malicious. But Merrill said one burgeoning area of growth for these phishing kits — fake e-commerce shops — can be far harder to spot because they do not call attention to themselves by spamming the entire world.

Merrill said the same Chinese phishing kits used to blast out package redelivery message scams are equipped with modules that make it simple to quickly deploy a fleet of fake but convincing e-commerce storefronts. Those phony stores are typically advertised on Google and Facebook, and consumers usually end up at them by searching online for deals on specific products.

A machine-translated screenshot of an ad from a China-based phishing group promoting their fake e-commerce shop templates.

With these fake e-commerce stores, the customer is supplying their payment card and personal information as part of the normal check-out process, which is then punctuated by a request for a one-time code sent by your financial institution. The fake shopping site claims the code is required by the user’s bank to verify the transaction, but it is sent to the user because the scammers immediately attempt to enroll the supplied card data in a mobile wallet.

According to Merrill, it is only during the check-out process that these fake shops will fetch the malicious code that gives them away as fraudulent, which tends to make it difficult to locate these stores simply by mass-scanning the web. Also, most customers who pay for products through these sites don’t realize they’ve been snookered until weeks later when the purchased item fails to arrive.

“The fake e-commerce sites are tough because a lot of them can fly under the radar,” Merrill said. “They can go months without being shut down, they’re hard to discover, and they generally don’t get flagged by safe browsing tools.”

Happily, reporting these SMS phishing lures and websites is one of the fastest ways to get them properly identified and shut down. Raymond Dijkxhoorn is the CEO and a founding member of SURBL, a widely-used blocklist that flags domains and IP addresses known to be used in unsolicited messages, phishing and malware distribution. SURBL has created a website called smishreport.com that asks users to forward a screenshot of any smishing message(s) received.

“If [a domain is] unlisted, we can find and add the new pattern and kill the rest” of the matching domains, Dijkxhoorn said. “Just make a screenshot and upload. The tool does the rest.”

The SMS phishing reporting site smishreport.com.

Merrill said the last few weeks of the calendar year typically see a big uptick in smishing — particularly package redelivery schemes that spoof the U.S. Postal Service or commercial shipping companies.

“Every holiday season there is an explosion in smishing activity,” he said. “Everyone is in a bigger hurry, frantically shopping online, paying less attention than they should, and they’re just in a better mindset to get phished.”

SHOP ONLINE LIKE A SECURITY PRO

As we can see, adopting a shopping strategy of simply buying from the online merchant with the lowest advertised prices can be a bit like playing Russian Roulette with your wallet. Even people who shop mainly at big-name online stores can get scammed if they’re not wary of too-good-to-be-true offers (think third-party sellers on these platforms).

If you don’t know much about the online merchant that has the item you wish to buy, take a few minutes to investigate its reputation. If you’re buying from an online store that is brand new, the risk that you will get scammed increases significantly. How do you know the lifespan of a site selling that must-have gadget at the lowest price? One easy way to get a quick idea is to run a basic WHOIS search on the site’s domain name. The more recent the site’s “created” date, the more likely it is a phantom store.

If you receive a message warning about a problem with an order or shipment, visit the e-commerce or shipping site directly, and avoid clicking on links or attachments — particularly missives that warn of some dire consequences unless you act quickly. Phishers and malware purveyors typically seize upon some kind of emergency to create a false alarm that often causes recipients to temporarily let their guard down.

But it’s not just outright scammers who can trip up your holiday shopping: Often times, items that are advertised at steeper discounts than other online stores make up for it by charging way more than normal for shipping and handling.

So be careful what you agree to: Check to make sure you know how long the item will take to be shipped, and that you understand the store’s return policies. Also, keep an eye out for hidden surcharges, and be wary of blithely clicking “ok” during the checkout process.

Most importantly, keep a close eye on your monthly statements. If I were a fraudster, I’d most definitely wait until the holidays to cram through a bunch of unauthorized charges on stolen cards, so that the bogus purchases would get buried amid a flurry of other legitimate transactions. That’s why it’s key to closely review your credit card bill and to quickly dispute any charges you didn’t authorize.

Top

FreeBSD 15: Why You’ll Want It

Post by FreeBSD Foundation via FreeBSD Foundation »

FreeBSD 15.0 landed earlier this week, and we read through the release notes with a fine-toothed comb to highlight some of the key improvements. Here are the standouts.

The BIG One: “pkgbase”

After roughly a decade of work, the base system can now be managed using pkg. Now that it’s here, what does pkgbase mean for everyday users? Drawing from a talk by Baptiste Daroussin, here are the key benefits:

  • Allows users to do fine grained installations (no toolchain, no debug symbols, etc.)
  • Offers more precise merging of configuration files.
  • Developers can easily ship packages for testing.
  • Permits simpler binary upgrades, including smoother tracking of STABLE and CURRENT.

This gives administrators much more flexibility in keeping systems minimal, consistent, and up to date.

Improved Desktop & Laptop Support

Several updates directly benefit laptop and desktop users:

Wi-Fi enhancements

  • The rtwn(4) driver now supports 802.11ac (VHT) for supported Realtek chipsets (RTL8812A and RTL8821A).
  • The new iwx(4) driver, FreeBSD’s native driver for newer Intel wireless chipsets, appears in this release as an alternative to iwlwifi(4).

Audio & device handling

  • Asynchronous device detach is now supported. This improves hot-plug behavior (e.g., USB headsets) and eases use of PulseAudio in cases that require operating system sleep/wake.

AMD GPU stability

  • Fixes landed for gradual slowdowns and freezes affecting certain AMD GPUs when using the amdgpu DRM driver from the drm-kmod ports package.

Offline Help for New Users

Brand new FreeBSD users often need guidance right after install, especially before their system is online.

To help:

  • The existing freebsd-base man page remains an invaluable starting point.
  • A new networking man page provides quick, offline guidance for troubleshooting early network setup.

This is extremely handy if a freshly installed system isn’t connecting to the network.

Major Improvements on Amazon Web Services

Running FreeBSD in AWS? You’ll notice some meaningful improvements:

  • FreeBSD “base” EC2 images now boot up to 76% faster than corresponding 14.0-RELEASE images, with the largest improvements found on arm64 (“Graviton”) instances.
  • The FreeBSD project now publishes “small” EC2 images; these are the “base” images minus debug symbols, tests, 32-bit libraries, the LLDB debugger, the Amazon SSM Agent, and the AWS CLI. This reduces the amount of disk space in use when the EC2 instance finishes booting from ~5 GB to ~1 GB. (wow!)
  • The FreeBSD project now publishes “builder” EC2 images; these boot into a memory disk and extract a clean “base” image onto the root disk (mounted at /mnt) to be customized before creating an AMI. 584265890303 (Sponsored by Amazon)

Other Noteworthy Updates

A few additional items that deserve attention:

  • Possibly bigger news than it appears – bhyve(8) and vmm(4) now support arm64 and riscv platforms.
  • FreeBSD introduces a native mechanism for controlled privilege escalation via mdo(1) and mac_do(4). This provides a built-in alternative to installing tools like sudo or doas when users need limited administrative capabilities.

But Wait! There’s More!

These highlights only scratch the surface. If you’re planning to upgrade—or just want a deeper look—reading the full FreeBSD 15.0 release notes is absolutely worthwhile.

The post FreeBSD 15: Why You’ll Want It first appeared on FreeBSD Foundation.

Top

Post by FreeBSD Newsflash via FreeBSD News Flash »

New committer: Timothy Pearson (src)
Top

How to recover when booting to multiple zroots?

Post by Dan Langille via Dan Langille's Other Diary »

On Nov 28, I rebooted r730-01 after an update, and it didn’t come back. At all.

I had inserted 2x 4TB SSDS a few days earlier. It seems they contained remnants of a zroot. The host was trying to boot from them. I couldn’t get the host to boot from the real zroot. It frustrated me.

I gave up and went to sleep later. The next morning, I moved those two drives into r730-04 and rebooted r730-01 – which recovered gracefully.

My goal now: reproduce the situation on r730-04 and see if I can resolve the problem from the console without removing the drives. And without booting a live boot image (e.g. thumbdrive).

In this post:

  • FreeBSD 14.3
  • mfsBSD 14.2

Does it boot?

No, it does not boot. I get this:

cannot boot
cannot boot

From here, I’m not sure what to do. My goal with this post: find a way to fix this without resorting to a live thumbdrive. That is, fix this from the console.

Is that possible?

What’s in the box?

When booting mfsBSD, I dropped into the loader prompted and entered this:

set hw.mfi.mrsas_enable="1"
boot

Then I ssh’d in as root (password is mfsroot).

From mfsBSD, I see this:

root@mfsbsd:~ # zpool import
   pool: zroot
     id: 17107398103472157640
  state: DEGRADED
status: One or more devices contains corrupted data.
 action: The pool can be imported despite missing or damaged devices.  The
	fault tolerance of the pool may be compromised if imported.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-4J
 config:

	zroot       DEGRADED
	  mirror-0  DEGRADED
	    da0p3   UNAVAIL  invalid label
	    da1p3   UNAVAIL  invalid label
	    da3p3   ONLINE

   pool: zroot_30G_drives
     id: 244204476102814311
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	zroot_30G_drives  ONLINE
	  mirror-0        ONLINE
	    da4p3         ONLINE
	    da5p3         ONLINE

   pool: data04
     id: 14613959245391720618
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	data04                           ONLINE
	  mirror-0                       ONLINE
	    diskid/DISK-S757NS0Y700758M  ONLINE
	    diskid/DISK-S757NS0Y700760R  ONLINE

   pool: zroot
     id: 16510343806874404582
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	zroot       ONLINE
	  mirror-0  ONLINE
	    da0p3   ONLINE
	    da1p3   ONLINE

   pool: zroot
     id: 16789754035394283994
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

	zroot       ONLINE
	  mirror-0  ONLINE
	    ada0p3  ONLINE
	    ada1p3  ONLINE
root@mfsbsd:~ # 

I know that last zroot, on ada0/1, is the one I want to boot from. I recognize the device names associated with the SATADOM devices attached directly to the m/b of the server.

root@mfsbsd:~ # grep ada0 /var/log/messages
Dec  2 17:54:30 mfsbsd kernel: ada0 at ahcich10 bus 0 scbus13 target 0 lun 0
Dec  2 17:54:30 mfsbsd kernel: ada0:  ACS-2 ATA SATA 3.x device
Dec  2 17:54:30 mfsbsd kernel: ada0: Serial Number 20170718AA0000185541
Dec  2 17:54:30 mfsbsd kernel: ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
Dec  2 17:54:30 mfsbsd kernel: ada0: Command Queueing enabled
Dec  2 17:54:30 mfsbsd kernel: ada0: 118288MB (242255664 512 byte sectors)
Dec  2 17:54:30 mfsbsd kernel: ses1: ada0,pass7 in 'Slot 04', SATA Slot: scbus13 target 0
root@mfsbsd:~ # grep ada1 /var/log/messages
Dec  2 17:54:30 mfsbsd kernel: ada1 at ahcich11 bus 0 scbus14 target 0 lun 0
Dec  2 17:54:30 mfsbsd kernel: ada1:  ACS-2 ATA SATA 3.x device
Dec  2 17:54:30 mfsbsd kernel: ada1: Serial Number 20170718AA0000185552
Dec  2 17:54:30 mfsbsd kernel: ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
Dec  2 17:54:30 mfsbsd kernel: ada1: Command Queueing enabled
Dec  2 17:54:30 mfsbsd kernel: ada1: 118288MB (242255664 512 byte sectors)
Dec  2 17:54:30 mfsbsd kernel: ses1: ada1,pass8 in 'Slot 05', SATA Slot: scbus14 target 0
root@mfsbsd:~ # 

For the zpools shown above, here are the partitions:

line 1 – zroot I don’t want

This is all messed up. Why is da3 in there? Oh wait, perhaps brought in by device name instead of gpt label?

root@mfsbsd:~ # gpart show da0 da1 da3
=>       40  468862048  da0  GPT  (224G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  464664576    3  freebsd-zfs  (222G)
  468860928       1160       - free -  (580K)

=>       40  468862048  da1  GPT  (224G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  464664576    3  freebsd-zfs  (222G)
  468860928       1160       - free -  (580K)

=>        34  7814037101  da3  GPT  (3.6T)
          34    17311710       - free -  (8.3G)
    17311744  1936213384    3  freebsd-zfs  (923G)
  1953525128  5860512007       - free -  (2.7T)

root@mfsbsd:~ # 

line 17 – zroot_30G_drives

This looks valid, but it’s not the zroot I want.d

root@mfsbsd:~ # gpart show da4 da5
=>      40  62533216  da4  GPT  (30G)
        40      2008       - free -  (1.0M)
      2048    409600    1  efi  (200M)
    411648  16777216    2  freebsd-swap  (8.0G)
  17188864  45344392    3  freebsd-zfs  (22G)

=>      40  62533216  da5  GPT  (30G)
        40      2008       - free -  (1.0M)
      2048    409600    1  efi  (200M)
    411648  16777216    2  freebsd-swap  (8.0G)
  17188864  45344392    3  freebsd-zfs  (22G)

root@mfsbsd:~ # 

line 28 – data04

Now I need to convert diskid/DISK-S757NS0Y700758M to a device name. For that, I used glabel list | less
and searched fort that text. I found this:

Geom name: da3
Providers:
1. Name: diskid/DISK-S757NS0Y700758M
   Mediasize: 4000787030016 (3.6T)

And this:

Geom name: da2
Providers:
1. Name: diskid/DISK-S757NS0Y700760R

Which leads to this command:

root@mfsbsd:~ # gpart show da3 da2
=>        34  7814037101  da3  GPT  (3.6T)
          34    17311710       - free -  (8.3G)
    17311744  1936213384    3  freebsd-zfs  (923G)
  1953525128  5860512007       - free -  (2.7T)

gpart: No such geom: da2.

line 39- zroot (for da3/da2) – I don’t want this one

gpart show da0 da1
root@mfsbsd:~ # gpart show da0 da1
=>       40  468862048  da0  GPT  (224G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  464664576    3  freebsd-zfs  (222G)
  468860928       1160       - free -  (580K)

=>       40  468862048  da1  GPT  (224G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  464664576    3  freebsd-zfs  (222G)
  468860928       1160       - free -  (580K)

line 50 – zroot ada0/ada1 – THIS is the zroot I want to boot from

root@mfsbsd:~ # gpart show ada0 ada1
=>       40  242255584  ada0  GPT  (116G)
         40     532480     1  efi  (260M)
     532520       2008        - free -  (1.0M)
     534528   33554432     2  freebsd-swap  (16G)
   34088960  208164864     3  freebsd-zfs  (99G)
  242253824       1800        - free -  (900K)

=>       40  242255584  ada1  GPT  (116G)
         40     532480     1  efi  (260M)
     532520       2008        - free -  (1.0M)
     534528   33554432     2  freebsd-swap  (16G)
   34088960  208164864     3  freebsd-zfs  (99G)
  242253824       1800        - free -  (900K)
Top

Reverse Engineering the SGI Indy Monitor Detection, or "thank god someone added SGI indy / indigo 2 support to MAME"

Post by Adrian Chadd via Adrian Chadd's Ramblings »

 I have a bit of a soft spot in my heart for the SGI Indy and (purple, not teal, heh) Indigo 2.

So imagine my surprise when NetBSD "almost" booted just fine on the Indy I have acquired. R4600PC-100, XL8 graphics .. and wonky console colours in netbsd / wonky xorg.

The first deep dive is "why are the monitor colours so unpredictable?" and that got me into a fun deep dive into how the SGI Indy Newport graphics works, the whole SGI Indy Linux project circa 2000, hardware shortcuts and software shortcuts.

Anyway.

The TL;DR is here - https://erikarn.github.io/sgi/indy/monitor_detection  . I've listed the monitor resolution/refresh rates the internet and my reverse engineering based on what MAME was programming.

So the long version.

First up - I've put all the hardware documentation I've found so far at https://erikarn.github.io/sgi/indy/notes

The Indy was booting NetBSD in either correct colours - green kernel text, white userland console text - or incorrect colours - green kernel text, but blue console text. It was random, and it was per boot. X11 was no better - sometimes it had the correct colours, sometimes everything was wonky.

The NetBSD console code tries to setup the following things for 8 bit graphics mode (which is used for console, even for 24 bit cards):

  • Program in an 256 entry colourmap table, matching what the NetBSD RGB 332 colour scheme is;
  • Add in a 1:1 RGB ramp in another colour table (RGB2);
  • A bunch of "XMAP9 mode" lines mapping 32 entries of "something" to RGB8 pixel format, RGB2 colourmap.

I was very confused as to what was and what should be going on, and I don't want to dig into the journey I took to get here. But the TL;DR here is that everything in the NetBSD console setup path is wrong and when it "worked", it ended up with the wrong colours. And when it "didn't work", it sometimes ended up with the wrong colours.

I'll write a separate post later about how the whole newport graphics system holds together, but fixing this requires a whole lot of driver changes to correctly program the hardware, and then some funky monitor timing specific programming.

The 13W3 port on the Newport graphics boards have a 4 bit monitor ID which compatible monitors will output. There's more details available at https://old.pinouts.ru/Video/sgivideo_pinout.shtml


The "universal 13W3 interface input cable" that I bought has a bunch of DIP switches controlling this.


 

If you have the four monitor ID bits set on or off, then you still get 1024x768 @ 60Hz.

The "fun" part of this story is if I were using 1280x1024 straight off the bat then I'd likely not have seen these problems happen so often.

Anyway.

Depending upon the settings, the Indy will boot with a bunch of different possible monitor setups:

  • 1024x768, 60Hz
  • 1024x768, 70Hz
  • 1280x1024, 60Hz
  • 1280x1024, 72Hz
  • 1280x1024, 76Hz

I enumerated this list and threw them up on the monitor detection link at the beginning of the article.

So, the firmware reads these four bits at boot (via 4 IO bits on one of the CMAP chips - again, see the links at the top of the post) sets up the monitor timing and then displays stuff. But when NetBSD's console programming is getting the colours wrong when I'm using 1024x768 60Hz.

It turns out that the XMAP chips - which handle the final mapping of incoming framebuffer pixel data to what 24 bit RGB is sent to the CMAP chip and then the RAMDAC -  were being programmed inconsistently. (again, they were being programmed incorrectly too in NetBSD, but I've got a big diff to fix that. With that, they're programmed correctly inconsistently.)

There's a "display control bus" that the Newport raster chip (REX3) has to peripheral chips. The peripheral chips - the XMAPs, the VC for timing, the RAMDAC, the CMAP for 8/24 bit colour table mapping - they're all DCB peripherals. The DCB has some address lines, 8 data bits, programmable chip select line, chip select setup, hold and release timing, optional request/ACK signaling, and register auto-increment functionality.

However!

  • The REX3 chip runs at 33MHz;
  • The XMAP chips run at 1/2 the pixel clock (they're interleaved);
  • The DCB has support for explicit ACK signaling from the peripheral, but as long as the peripheral uses it;
  • The XMAP does not have an ACK line, just an incoming chip select line, and
  • When writing the XMAP mode table lines - which map the display information to pixel format / colour table selection - it's done as back to back bursts to the same register, not an auto-increment and NOT using an ACK line.

This means that if the XMAP chip is running at a speed that doesn't entirely line up with the programmed chipselect timing, the mode writes will be junk. The normal 8 bit read/writes are "mostly fine" as they just show up as multiple 8 bit read/writes to the same register and for all the OTHER registers that is just fine. But for the mode register - where the DCB needs to write 4 bytes to the same individual address - it's absolutely not fine.

And that's the kicker.

After spending some quality time with the MAME emulator and some local hacks to enable the newport peripheral IO logging and setting the monitor ID, I found out that the timing used for the XMAP chips is different for 1024x768 60Hz versus 1280x1024 76Hz.

Everything worked just fine when I adjusted it.

So ok, I went back to the Linux and X11 drivers to see what's going on there, as I know the C code wasn't doing this. And I found this gem in the Linux newport.h header file:

 static __inline__ void
xmap9SetModeReg (struct newport_regs *rex, unsigned int modereg, unsigned int data24, int cfreq)
{
        if (cfreq > 119)
            rex->set.dcbmode = DCB_XMAP_ALL | XM9_CRS_MODE_REG_DATA |
                        DCB_DATAWIDTH_4 | W_DCB_XMAP9_PROTOCOL;
        else if (cfreq > 59)
            rex->set.dcbmode = DCB_XMAP_ALL | XM9_CRS_MODE_REG_DATA |
                    DCB_DATAWIDTH_4 | WSLOW_DCB_XMAP9_PROTOCOL;
        else
            rex->set.dcbmode = DCB_XMAP_ALL | XM9_CRS_MODE_REG_DATA |
                        DCB_DATAWIDTH_4 | WAYSLOW_DCB_XMAP9_PROTOCOL;
        rex->set.dcbdata0.byword = ((modereg) << 24) | (data24 & 0xffffff);
}

It's choosing different DCB timing based on the pixel clock. It lines up with what I've been seeing from MAME and it adds a third one - WAYSLOW - which I bet I'm only going to see on the PAL/NTSC timings or if something really wants to do something like 1024x768 50Hz.

The timings are in the header file, but .. nothing is using xmap9setModeReg(). It was likely copied from some internal SGI code (the PROM? X server? Who knows!) as part of the code bring-up but it was never used.

Anyway! With this in the NetBSD console code the console finally works reliably in all the modes I've tested. I'm going to try and get my big diff stack landed in NetBSD and then I'll work on the X11 newport code so it too supports 8 and 24 bit graphics at 1024x768 reliably.

  • Read the CMAP1 register (and PROM on SGI Indy) to determine the monitor type
  • The default monitor on SGI Indy is 1024x768 60Hz, and for Indigo2 its 1280x1024 60Hz
  • Select an XMAP9 mode DCB timing set based on the pixel clock
  • 8 bit mode for console and X11 needs the colour index table programmed into the CMAP at CI offset 0, and appropriate XMAP config for the display mode table to use 8 bit pixels, PIXMODE_CI, offset 0, NOT 8 bit RGB
  • 24 bit mode for X11 needs the 24 bit RGB ramp programmed into the CMAP RGB2 table (which is not a colour index table!), and no CMAP
  • Importantly, the X11 server uses truecolour for 24 bit mode, and pseudocolour / colourmaps for 8 bit mode, so all of this need repeating in the X11 server code! 

Here's how the console looks, complete with the correct XMAP9 mode table:

And here's how x11 looks:


 

 

(And the X11 support is even more fun, because I had to fix some stuff to make acceleration in the driver work, and that's going to be a whole fun DIFFERENT post.)

Addendum:

Oh, and the sync on green? It's generated by the RAMDAC. Once this all has landed in NetBSD I'm tempted to try to add a sysctl / boot parameter to disable the sync on green bit so normal monitors work on the SGI Indy. Let me know if you'd like that! 

Top