Planet 2014-09-16 09:00 UTC


One of the risks of using social media networks is having information you intend to share with only a handful of friends be made available to everyone. Sometimes that over-sharing happens because friends betray your trust, but more worrisome are the cases in which a social media platform itself exposes your data in the name of marketing.

leakedinlogoLinkedIn has built much of its considerable worth on the age-old maxim that “it’s all about who you know”: As a LinkedIn user, you can directly connect with those you attest to knowing professionally or personally, but also you can ask to be introduced to someone you’d like to meet by sending a request through someone who bridges your separate social networks. Celebrities, executives or any other LinkedIn users who wish to avoid unsolicited contact requests may do so by selecting an option that forces the requesting party to supply the personal email address of the intended recipient.

LinkedIn’s entire social fabric begins to unravel if any user can directly connect to any other user, regardless of whether or how their social or professional circles overlap. Unfortunately for LinkedIn (and its users who wish to have their email addresses kept private), this is the exact risk introduced by the company’s built-in efforts to expand the social network’s user base.

According to researchers at the Seattle, Wash.-based firm Rhino Security Labs, at the crux of the issue is LinkedIn’s penchant for making sure you’re as connected as you possibly can be. When you sign up for a new account, for example, the service asks if you’d like to check your contacts lists at other online services (such as Gmail, Yahoo, Hotmail, etc.). The service does this so that you can connect with any email contacts that are already on LinkedIn, and so that LinkedIn can send invitations to your contacts who aren’t already users.

LinkedIn assumes that if an email address is in your contacts list, that you must already know this person. But what if your entire reason for signing up with LinkedIn is to discover the private email addresses of famous people? All you’d need to do is populate your email account’s contacts list with hundreds of permutations of famous peoples’ names — including combinations of last names, first names and initials — in front of @gmail.com, @yahoo.com, @hotmail.com, etc. With any luck and some imagination, you may well be on your way to an A-list LinkedIn friends list (or a fantastic set of addresses for spear-phishing, stalking, etc.).

LinkedIn lets you know which of your contacts aren't members.

LinkedIn lets you know which of your contacts aren’t members.

When you import your list of contacts from a third-party service or from a stand-alone file, LinkedIn will show you any profiles that match addresses in your contacts list. More significantly, LinkedIn helpfully tells you which email addresses in your contacts lists are not LinkedIn users.

It’s that last step that’s key to finding the email address of the targeted user to whom LinkedIn has just sent a connection request on your behalf. The service doesn’t explicitly tell you that person’s email address, but by comparing your email account’s contact list to the list of addresses that LinkedIn says don’t belong to any users, you can quickly figure out which address(es) on the contacts list correspond to the user(s) you’re trying to find.

Rhino Security founders Benjamin Caudill and Bryan Seely have a recent history of revealing how trust relationships between and among online services can be abused to expose or divert potentially sensitive information. Last month, the two researchers detailed how they were able to de-anonymize posts to Secret, an app-driven online service that allows people to share messages anonymously within their circle of friends, friends of friends, and publicly. In February, Seely more famously demonstrated how to use Google Maps to intercept FBI and Secret Service phone calls.

This time around, the researchers picked on Dallas Mavericks owner Mark Cuban to prove their point with LinkedIn. Using their low-tech hack, the duo was able to locate the Webmail address Cuban had used to sign up for LinkedIn. Seely said they found success in locating the email addresses of other celebrities using the same method about nine times out ten.

“We created several hundred possible addresses for Cuban in a few seconds, using a Microsoft Excel macro,” Seely said. “It’s just a brute-force guessing game, but 90 percent of people are going to use an email address that includes components of their real name.”

The Rhino guys really wanted Cuban’s help in spreading the word about what they’d found, but instead of messaging Cuban directly, Seely pursued a more subtle approach: He knew Cuban’s latest start-up was Cyber Dust, a chat messenger app designed to keep your messages private. So, Seely fired off a tweet complaining that “Facebook Messenger crosses all privacy lines,” and that as  result he was switching to Cyber Dust.

When Mark Cuban retweeted Seely’s endorsement of Cyber Dust, Seely reached out to Cyberdust CEO Ryan Ozonian, letting him know that he’d discovered Cuban’s email address on LinkedIn. In short order, Cuban was asking Rhino to test the security of Cyber Dust.

“Fortunately no major faults were found and those he found are already fixed in the coming update,” Cuban said in an email exchange with KrebsOnSecurity. “I like working with them. They look to help rather than exploit.. We have learned from them and I think their experience will be valuable to other app publishers and networks as well.”

Cory Scott, director of information security at LinkedIn, said very few of the company’s members opt-in to the requirement that all new potential contacts supply the invitee’s email address before sending an invitation to connect. He added that email address-to-user mapping is a fairly common design pattern, and that is is not particularly unique to LinkedIn, and that nothing the company does will prevent people from blasting emails to lists of addresses that might belong to a targeted user, hoping that one of them will hit home.

“Email address permutators, of which there are many of them on the ‘Net, have existed much longer than LinkedIn, and you can blast an email to all of them, knowing that most likely one of those will hit your target,” Scott said. “This is kind of one of those challenges that all social media companies face in trying to prevent the abuse of [site] functionality. We have rate limiting, scoring and abuse detection mechanisms to prevent frequent abusers of this service, and to make sure that people can’t validate spam lists.”

In an email sent to this reporter last week, LinkedIn said it was planning at least two changes to the way its service handles user email addresses.

“We are in the process of implementing two short-term changes and one longer term change to give our members more control over this feature,” Linkedin spokeswoman Nicole Leverich wrote in an emailed statement. “In the next few weeks, we are introducing new logic models designed to prevent hackers from abusing this feature. In addition, we are making it possible for members to ask us to opt out of being discoverable through this feature. In the longer term, we are looking into creating an opt-out box that members can choose to select to not be discoverable using this feature.”



The first BETA build of the 10.1-RELEASE release cycle is now available on the FTP servers for the amd64, armv6, i386, ia64, powerpc, powerpc64 and sparc64 architectures.

The image checksums follow are included in the original announcement email.

Installer images and memory stick images are available here.

If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list.

If you would like to use SVN to do a source based update of an existing system, use the "stable/10" branch.

A list of changes since 10.0-RELEASE are available on the stable/10 release notes page.

Pre-installed virtual machine images for 10.1-BETA1 are also available for amd64 and i386 architectures.  The images are located here.

The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats.  The image download size is approximately 135 MB, which decompress to a 20GB sparse image.

The partition layout is:
  • 512k - freebsd-boot GPT partition type (bootfs GPT label)
  • 1GB  - freebsd-swap GPT partition type (swapfs GPT label)
  • ~17GB - freebsd-ufs GPT partition type (rootfs GPT label)
Note to consumers of the dvd1.iso image: The packages included on the dvd do not have a corresponding pkg(8) repository due to an incompatibility with pkg-1.2.x and pkg-1.3.x.  This will be fixed for BETA2.

The packages will not be recognized by bsdconfig(8), however can be  installed manually.

To install packages from the dvd1.iso installer, create and mount the /dist directory:

# mkdir -p /dist
# mount -t cd9660 /dev/cd0 /dist

Next, install pkg(8) from the DVD:
 

# env REPOS_DIR=/dist/packages/repos pkg add \
  /dist/packages/freebsd:10:*:*/All/pkg-*.txz

At this point, pkg-add(8) can be used to install additional packages from the DVD.  Please note, the REPOS_DIR environment variable should be used each time using the DVD as the package repository, otherwise conflicts with packages from the upstream mirrors may occur when they are fetched.  For example, to install the Subversion, Gnome, and Xorg, run:
 

# env REPOS_DIR=/dist/packages/repos pkg add \
  /dist/packages/freebsd:10:*:*/subversion [...]

The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases.  Systems running earlier
FreeBSD releases can upgrade as follows:

# freebsd-update upgrade -r 10.1-BETA1

During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically
performed merging was done correctly.

# freebsd-update install

The system must be rebooted with the newly installed kernel before continuing.


# shutdown -r now

After rebooting, freebsd-update needs to be run again to install the new userland components:


# freebsd-update install
It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example,
FreeBSD 8.x.  Alternatively, the user can install misc/compat9x and other compatibility libraries, afterwards the system must be rebooted
into the new userland:

# shutdown -r now

Finally, after rebooting, freebsd-update needs to be run again to remove stale files:

# freebsd-update install

Love FreeBSD?  Support this and future releases with a donation to the FreeBSD Foundation!



After some initially positive experience with ghc-7.8-rc1 I’ve decided to upstream most of gentoo fixes.

On rare arches ghc-7.8.3 behaves a bit bad:

  • ia64 build stopped being able to link itself after ghc-7.4 (gprel overflow)
  • on sparc, ia64 and ppc ghc was not able to create working shared libraries
  • integer-gmp library on ia64 crashed, and we had to use integer-simple

I have written a small story of those fixes here if you are curious.

TL;DR:

To get ghc-7.8.3 working nicer for exotic arches you will need to backport at least the following patches:

Thank you!




I think it's about time I shared some more details about the RSS stuff going into FreeBSD and how I'm testing it.

For now I'm focusing on IPv4 + UDP on the Intel 10GE NICs. The TCP side of things is done (and the IPv6 side of things works too!) but enough of the performance walls show up in the IPv4 UDP case that it's worth sticking to it for now.

I'm testing on a pair of 4-core boxes at home. They're not special - and they're very specifically not trying to be server-class hardware. I'd like to see where these bottlenecks are even at low core count.

The test setup in question:

Testing software:

  • http://github.com/erikarn/freebsd-rss
  • It requires libevent2 - an updated copy; previous versions of libevent2 didn't handle FreeBSD specific errors gracefully and would early error out of the IO loop.

Server:

  • CPU: Intel(R) Core(TM) i5-3550 CPU @ 3.30GHz (3292.59-MHz K8-class CPU)
  • There's no SMT/HTT, but I've disabled it in the BIOS just to be sure
  • 4GB RAM
  • FreeBSD-HEAD, amd64
  • NIC:  '82599EB 10-Gigabit SFI/SFP+ Network Connection
  • ix0: 10.11.2.1/24
/etc/sysctl.conf:

# for now redirect processing just makes the lock overhead suck even more.
# disable it.
net.inet.ip.redirect=0
net.inet.icmp.drop_redirect=1

/boot/loader.conf:

hw.ix.num_queues=8

# experiment with deferred dispatch for RSS
net.isr.numthreads=4
net.isr.maxthreads=4
net.isr.bindthreads=1
 

kernel config:

include GENERIC
ident HACKBOX

device netmap
options RSS
options PCBGROUP

# in-system lock profiling
options LOCK_PROFILING

# Flowtable - the rtentry locking is a bit .. slow.
options   FLOWTABLE

# This debugging code has too much overhead to do accurate
# testing with.
nooptions         INVARIANTS
nooptions         INVARIANT_SUPPORT
nooptions         WITNESS
nooptions         WITNESS_SKIPSPIN


The server runs the "rss-udp-srv" process, which behaves like a multi-threaded UDP echo server on port 8080.

Client

The client box is slightly more powerful to compensate for (currently) not using completely affinity-aware RSS UDP transmit code.

  • CPU: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz (3192.68-MHz K8-class CPU)
  • SMT/HTT: Disabled in BIOS
  • 8GB RAM
  • FreeBSD-HEAD amd64
  • Same kernel config, loader and sysctl config as the server
  • ix0: configured as 10.11.2.2/24, 10.11.2.3/32, 10.11.2.4/32, 10.11.2.32/32, 10.11.2.33/32
The client runs 'udp-clt' programs to source and sink traffic to the server.

Running things

The server-side simply runs the listen server, configured to respond to each frame:

$ rss-udp-srv 1 10.11.2.1

The client-side runs four couples of udp-clt, each from different IP addresses. These are run in parallel (i do it in different screens, so I can quickly see what's going on):

$ ./udp-clt -l 10.11.2.3 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.4 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.32 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.33 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510

The IP addresses are chosen so that the 2-tuple topelitz hash using the default Microsoft key hash to different RSS buckets that live on individual CPUs.

Results: Round one

When the server is responding to each frame, the following occurs. The numbers are "number of frames generated by the client (netstat)", "number of frames received by the server (netstat)", "number of frames seen by udp-rss-srv", "number of responses transmitted from udp-rss-srv", "number of frames seen by the server (netstat)"
  • 1 udp-clt process: 710,000; 710,000; 296,000; 283,000; 281,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 592,000; 592,000; 575,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 636,000; 636,000; 600,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 255,000; 255,000; 255,000
So, it's not actually linear past two cores. The question here is: why?

There are a couple of parts to this.

Firstly - I had left turbo boost on. What this translated to:

  • One core active: ~ 30% increase in clock speed
  • Two cores active: ~ 30% increase in clock speed
  • Three cores active: ~ 25% increase in clock speed
  • Four cores active: ~ 15% increase in clock speed.
Secondly and more importantly - I had left flow control enabled. This made a world of difference.

The revised results are mostly linear - with more active RSS buckets (and thus CPUs) things seem to get slightly more efficient:
  • 1 udp-clt process: 710,000; 710,000; 266,000; 266,000; 266,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 512,000; 512,000; 512,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 810,000; 810,000; 810,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 1,120,000; 1,120,000; 1,120,000

Finally, let's repeat the process but only receiving instead also echoing back the packet to the client:

$ rss-udp-srv 0 10.11.2.1
  • 1 udp-clt process: 710,000; 710,000; 204,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 378,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 645,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 900,000
The receive-only workload is actually worse off versus the transmit + receive workload!

What's going on here?

Well, a little digging shows that in both instances - even with a single udp-clt thread running which means only one CPU on the server side is actually active! - there's active lock contention.

Here's an example dtrace output for measuring lock contention with only one active process, where one CPU is involved (and the other three are idle):

Receive only, 5 seconds:

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # dtrace -n 'lockstat:::adaptive-block { @[stack()] = sum(arg1); }'
dtrace: description 'lockstat:::adaptive-block ' matched 1 probe
^C


              kernel`udp_append+0x11c
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
         46729281

Transmit + receive, 5 seconds:

dtrace: description 'lockstat:::adaptive-block ' matched 1 probe
^C


              kernel`knote+0x7e
              kernel`sowakeup+0x65
              kernel`udp_append+0x14a
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
             3793

              kernel`udp_append+0x11c
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
          3823793

              kernel`ixgbe_msix_que+0xd3
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
          9918140

Somehow it seems there's less lock contention / blocking going on when both transmit and receive is running!

So then I dug into it using the lock profiling suite. This is for 5 seconds with receive-only traffic on a single RSS bucket / CPU (all other CPUs are idle):

# sysctl debug.lock.prof.enable = 1; sleep 5 ; sysctl debug.lock.prof.enable=0

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # sysctl debug.lock.prof.enable=1 ; sleep 5 ; sysctl debug.lock.prof.enable=0
debug.lock.prof.enable: 1 -> 1
debug.lock.prof.enable: 1 -> 0

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # sysctl debug.lock.prof.stats | head -2 ; sysctl debug.lock.prof.stats | sort -nk4 | tail -10
debug.lock.prof.stats: 
     max  wait_max       total  wait_total       count    avg wait_avg cnt_hold cnt_lock name
    1496         0       10900           0          28    389      0  0      0 /usr/home/adrian/work/freebsd/head/src/sys/dev/usb/usb_device.c:2755 (sx:USB config SX lock)
debug.lock.prof.stats: 
       0         0          31           1          67      0      0  0      4 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:888 (spin mutex:sched lock 2)
       0         0        2715           1       49740      0      0  0      7 /usr/home/adrian/work/freebsd/head/src/sys/dev/random/random_harvestq.c:294 (spin mutex:entropy harvest mutex)
       1         0          51           1         131      0      0  0      2 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:1179 (spin mutex:sched lock 1)
       0         0          69           2         170      0      0  0      8 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:886 (spin mutex:sched lock 2)
       0         0       40389           2      287649      0      0  0      8 /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1359 (spin mutex:sched lock 2)
       0         2           2           4          12      0      0  0      2 /usr/home/adrian/work/freebsd/head/src/sys/dev/usb/usb_device.c:2762 (sleep mutex:Giant)
      15        20        6556         520        2254      2      0  0    105 /usr/home/adrian/work/freebsd/head/src/sys/dev/acpica/Osd/OsdSynch.c:535 (spin mutex:ACPI lock (0xfffff80002b10f00))
       4         5      195967       65888     3445501      0      0  0  28975 /usr/home/adrian/work/freebsd/head/src/sys/netinet/udp_usrreq.c:369 (sleep mutex:so_rcv)

Notice the lock contention for the so_rcv (socket receive buffer) handling? What's going on here is pretty amusing - it turns out that because there's so much receive traffic going on, the userland process receiving the data is being preempted by the NIC receive thread very often - and when this happens, there's a good chance it's going to be within the small window that the receive socket buffer lock is held. Once this happens, the NIC receive thread processes frames until it gets to one that requires it to grab the same sock buffer lock that is already held by userland - and it fails - so the NIC thread sleeps until the userland thread finishes consuming a packet. Then the CPU flips back to the NIC thread and continues processing a packet.

When the userland code is also transmitting frames it's increasing the amount of time in between socket receives and decreasing the probability of hitting the lock contention condition above.

Note there's no contention between CPUs here - this is entirely contention within a single CPU.

So for now I'm happy that the UDP IPv4 path is scaling well enough with RSS on a single core. The main performance problem here is the socket receive buffer locking (and, yes, copyin() / copyout().)

Next!



I think it's about time I shared some more details about the RSS stuff going into FreeBSD and how I'm testing it.

For now I'm focusing on IPv4 + UDP on the Intel 10GE NICs. The TCP side of things is done (and the IPv6 side of things works too!) but enough of the performance walls show up in the IPv4 UDP case that it's worth sticking to it for now.

I'm testing on a pair of 4-core boxes at home. They're not special - and they're very specifically not trying to be server-class hardware. I'd like to see where these bottlenecks are even at low core count.

The test setup in question:

Testing software:

  • http://github.com/erikarn/freebsd-rss
  • It requires libevent2 - an updated copy; previous versions of libevent2 didn't handle FreeBSD specific errors gracefully and would early error out of the IO loop.

Server:

  • CPU: Intel(R) Core(TM) i5-3550 CPU @ 3.30GHz (3292.59-MHz K8-class CPU)
  • There's no SMT/HTT, but I've disabled it in the BIOS just to be sure
  • 4GB RAM
  • FreeBSD-HEAD, amd64
  • NIC:  '82599EB 10-Gigabit SFI/SFP+ Network Connection
  • ix0: 10.11.2.1/24
/etc/sysctl.conf:

# for now redirect processing just makes the lock overhead suck even more.
# disable it.
net.inet.ip.redirect=0
net.inet.icmp.drop_redirect=1

/boot/loader.conf:

hw.ix.num_queues=8

# experiment with deferred dispatch for RSS
net.isr.numthreads=4
net.isr.maxthreads=4
net.isr.bindthreads=1
 

kernel config:

include GENERIC
ident HACKBOX

device netmap
options RSS
options PCBGROUP

# in-system lock profiling
options LOCK_PROFILING

# Flowtable - the rtentry locking is a bit .. slow.
options   FLOWTABLE

# This debugging code has too much overhead to do accurate
# testing with.
nooptions         INVARIANTS
nooptions         INVARIANT_SUPPORT
nooptions         WITNESS
nooptions         WITNESS_SKIPSPIN


The server runs the "rss-udp-srv" process, which behaves like a multi-threaded UDP echo server on port 8080.

Client

The client box is slightly more powerful to compensate for (currently) not using completely affinity-aware RSS UDP transmit code.

  • CPU: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz (3192.68-MHz K8-class CPU)
  • SMT/HTT: Disabled in BIOS
  • 8GB RAM
  • FreeBSD-HEAD amd64
  • Same kernel config, loader and sysctl config as the server
  • ix0: configured as 10.11.2.2/24, 10.11.2.3/32, 10.11.2.4/32, 10.11.2.32/32, 10.11.2.33/32
The client runs 'udp-clt' programs to source and sink traffic to the server.

Running things

The server-side simply runs the listen server, configured to respond to each frame:

$ rss-udp-srv 1 10.11.2.1

The client-side runs four couples of udp-clt, each from different IP addresses. These are run in parallel (i do it in different screens, so I can quickly see what's going on):

$ ./udp-clt -l 10.11.2.3 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.4 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.32 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510
$ ./udp-clt -l 10.11.2.33 -r 10.11.2.1 -p 8080 -n 10000000000 -s 510

The IP addresses are chosen so that the 2-tuple topelitz hash using the default Microsoft key hash to different RSS buckets that live on individual CPUs.

Results: Round one

When the server is responding to each frame, the following occurs. The numbers are "number of frames generated by the client (netstat)", "number of frames received by the server (netstat)", "number of frames seen by udp-rss-srv", "number of responses transmitted from udp-rss-srv", "number of frames seen by the server (netstat)"
  • 1 udp-clt process: 710,000; 710,000; 296,000; 283,000; 281,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 592,000; 592,000; 575,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 636,000; 636,000; 600,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 255,000; 255,000; 255,000
So, it's not actually linear past two cores. The question here is: why?

There are a couple of parts to this.

Firstly - I had left turbo boost on. What this translated to:

  • One core active: ~ 30% increase in clock speed
  • Two cores active: ~ 30% increase in clock speed
  • Three cores active: ~ 25% increase in clock speed
  • Four cores active: ~ 15% increase in clock speed.
Secondly and more importantly - I had left flow control enabled. This made a world of difference.

The revised results are mostly linear - with more active RSS buckets (and thus CPUs) things seem to get slightly more efficient:
  • 1 udp-clt process: 710,000; 710,000; 266,000; 266,000; 266,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 512,000; 512,000; 512,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 810,000; 810,000; 810,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 1,120,000; 1,120,000; 1,120,000

Finally, let's repeat the process but only receiving instead also echoing back the packet to the client:

$ rss-udp-srv 0 10.11.2.1
  • 1 udp-clt process: 710,000; 710,000; 204,000
  • 2 udp-clt processes: 1,300,000; 1,300,000; 378,000
  • 3 udp-clt processes: 1,800,000; 1,800,000; 645,000
  • 4 udp-clt processes: 2,100,000; 2,100,000; 900,000
The receive-only workload is actually worse off versus the transmit + receive workload!

What's going on here?

Well, a little digging shows that in both instances - even with a single udp-clt thread running which means only one CPU on the server side is actually active! - there's active lock contention.

Here's an example dtrace output for measuring lock contention with only one active process, where one CPU is involved (and the other three are idle):

Receive only, 5 seconds:

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # dtrace -n 'lockstat:::adaptive-block { @[stack()] = sum(arg1); }'
dtrace: description 'lockstat:::adaptive-block ' matched 1 probe
^C


              kernel`udp_append+0x11c
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
         46729281

Transmit + receive, 5 seconds:

dtrace: description 'lockstat:::adaptive-block ' matched 1 probe
^C


              kernel`knote+0x7e
              kernel`sowakeup+0x65
              kernel`udp_append+0x14a
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
             3793

              kernel`udp_append+0x11c
              kernel`udp_input+0x8cc
              kernel`ip_input+0x116
              kernel`netisr_dispatch_src+0x1cb
              kernel`ether_demux+0x123
              kernel`ether_nh_input+0x34d
              kernel`netisr_dispatch_src+0x61
              kernel`ether_input+0x26
              kernel`ixgbe_rxeof+0x2f7
              kernel`ixgbe_msix_que+0xb6
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
          3823793

              kernel`ixgbe_msix_que+0xd3
              kernel`intr_event_execute_handlers+0x83
              kernel`ithread_loop+0x96
              kernel`fork_exit+0x71
              kernel`0xffffffff80cd19de
          9918140

Somehow it seems there's less lock contention / blocking going on when both transmit and receive is running!

So then I dug into it using the lock profiling suite. This is for 5 seconds with receive-only traffic on a single RSS bucket / CPU (all other CPUs are idle):

# sysctl debug.lock.prof.enable = 1; sleep 5 ; sysctl debug.lock.prof.enable=0

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # sysctl debug.lock.prof.enable=1 ; sleep 5 ; sysctl debug.lock.prof.enable=0
debug.lock.prof.enable: 1 -> 1
debug.lock.prof.enable: 1 -> 0

root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss # sysctl debug.lock.prof.stats | head -2 ; sysctl debug.lock.prof.stats | sort -nk4 | tail -10
debug.lock.prof.stats: 
     max  wait_max       total  wait_total       count    avg wait_avg cnt_hold cnt_lock name
    1496         0       10900           0          28    389      0  0      0 /usr/home/adrian/work/freebsd/head/src/sys/dev/usb/usb_device.c:2755 (sx:USB config SX lock)
debug.lock.prof.stats: 
       0         0          31           1          67      0      0  0      4 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:888 (spin mutex:sched lock 2)
       0         0        2715           1       49740      0      0  0      7 /usr/home/adrian/work/freebsd/head/src/sys/dev/random/random_harvestq.c:294 (spin mutex:entropy harvest mutex)
       1         0          51           1         131      0      0  0      2 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:1179 (spin mutex:sched lock 1)
       0         0          69           2         170      0      0  0      8 /usr/home/adrian/work/freebsd/head/src/sys/kern/sched_ule.c:886 (spin mutex:sched lock 2)
       0         0       40389           2      287649      0      0  0      8 /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1359 (spin mutex:sched lock 2)
       0         2           2           4          12      0      0  0      2 /usr/home/adrian/work/freebsd/head/src/sys/dev/usb/usb_device.c:2762 (sleep mutex:Giant)
      15        20        6556         520        2254      2      0  0    105 /usr/home/adrian/work/freebsd/head/src/sys/dev/acpica/Osd/OsdSynch.c:535 (spin mutex:ACPI lock (0xfffff80002b10f00))
       4         5      195967       65888     3445501      0      0  0  28975 /usr/home/adrian/work/freebsd/head/src/sys/netinet/udp_usrreq.c:369 (sleep mutex:so_rcv)

Notice the lock contention for the so_rcv (socket receive buffer) handling? What's going on here is pretty amusing - it turns out that because there's so much receive traffic going on, the userland process receiving the data is being preempted by the NIC receive thread very often - and when this happens, there's a good chance it's going to be within the small window that the receive socket buffer lock is held. Once this happens, the NIC receive thread processes frames until it gets to one that requires it to grab the same sock buffer lock that is already held by userland - and it fails - so the NIC thread sleeps until the userland thread finishes consuming a packet. Then the CPU flips back to the NIC thread and continues processing a packet.

When the userland code is also transmitting frames it's increasing the amount of time in between socket receives and decreasing the probability of hitting the lock contention condition above.

Note there's no contention between CPUs here - this is entirely contention within a single CPU.

So for now I'm happy that the UDP IPv4 path is scaling well enough with RSS on a single core. The main performance problem here is the socket receive buffer locking (and, yes, copyin() / copyout().)

Next!



After an excruciating wait and years of learning PostgreSQL, it’s time to unify the PostgreSQL ebuilds.I’m not sure what the original motivation was to split the ebuilds, but, from the history I’ve seen on Gentoo, it has always been that way. That’s a piss-poor reason for continuing to do things a certain way. Especially when that way is wrong and makes things more tedious and difficult than they ought to be.

I’m to blame for pressing forward with the splitting the ebuilds to -docs, -base, and -server when I first got started in Gentoo. I knew from the outset that having them split was not a good idea. I just didn’t know as much as I do now to defend one way or the other. To be fair, Patrick (bonsaikitten) would have gone with whatever I decided to do, but I thought I understood the advantages. Now I look at it and just see disadvantages.

Let’s first look at the build times for building the split ebuilds:

1961.35user 319.42system 33:59.44elapsed 111%CPU (0avgtext+0avgdata 682896maxresident)k
46696inputs+2000640outputs (34major+34350937minor)pagefaults 0swaps
1955.12user 325.01system 33:39.86elapsed 112%CPU (0avgtext+0avgdata 682896maxresident)k
7176inputs+1984960outputs (33major+34349678minor)pagefaults 0swaps
1942.90user 318.89system 33:53.70elapsed 111%CPU (0avgtext+0avgdata 682928maxresident)k
28496inputs+1999688outputs (124major+34343901minor)pagefaults 0swaps

And now the unified ebuild:

1823.57user 278.96system 30:29.20elapsed 114%CPU (0avgtext+0avgdata 683024maxresident)k
32520inputs+1455688outputs (100major+30199771minor)pagefaults 0swaps
1795.63user 282.55system 30:35.92elapsed 113%CPU (0avgtext+0avgdata 683024maxresident)k
9848inputs+1456056outputs (30major+30225195minor)pagefaults 0swaps
1802.47user 275.66system 30:08.30elapsed 114%CPU (0avgtext+0avgdata 683056maxresident)k
13800inputs+1454880outputs (49major+30193986minor)pagefaults 0swaps

So, the unified ebuild is about 10% faster than the split ebuilds.

There are also a few bugs open that will be resolved by moving to a unified ebuild. Whenever someone changes anything in their flags, Portage tends to only pick up dev-db/postgresql-server as needing to be recompiled rather than the appropriate dev-db/postgresql-base, which results in broken setups and failures to even build. I’ve even been accused of pulling the rug out from under people. I swear, it’s not me…it’s Portage…who I lied to. Kind of.

There should be little interruption, though, to the end user. I’ll be keeping all the features that splitting brought us. Okay, feature. There’s really just one feature: Proper slotting. Portage will be informed of the package moves, and everything should be hunky-dory with one caveat: A new ‘server’ USE flag is being introduced to control whether to build everything or just the clients and libraries.

No, I don’t want to do a lib-only flag. I don’t want to work on another hack.

You can check out the progress on my overlay. I’ll be working on the updating the dependent packages as well so they’re all ready to go in one shot.



Adobe today released updates to fix at least a dozen critical security problems in its Flash Player and AIR software. Separately, Microsoft pushed four update bundles to address at least 42 vulnerabilities in Windows, Internet Explorer, Lync and .NET Framework. If you use any of these, it’s time to update!

winiconMost of the flaws Microsoft fixed today (37 of them) are addressed in an Internet Explorer update — the only patch this month to earn Microsoft’s most-dire “critical” label. A critical update wins that rating if the vulnerabilities fixed in the update could be exploited with little to no action on the part of users, save for perhaps visiting a hacked or malicious Web site with IE.

I’ve experienced troubles installing Patch Tuesday packages along with .NET updates, so I make every effort to update .NET separately. To avoid any complications, I would recommend that Windows users install all other available recommended patches except for the .NET bundle; after installing those updates, restart Windows and then install any pending .NET fixes). Your mileage may vary.

For more information on the rest of the updates released today, see this post at the Microsoft Security Response Center Blog.

brokenflash-aAdobe’s critical update for Flash Player fixes at least 12 security holes in the program. Adobe is urging Windows and Macintosh users to update to Adobe Flash Player v. 15.0.0.152 by visiting the Adobe Flash Player Download Center, or via the update mechanism within the product when prompted. If you’d rather not be bothered with downloaders and software “extras” like antivirus scanners, you’re probably best off getting the appropriate update for your operating system from this link.

To see which version of Flash you have installed, check this link. IE10/IE11 on Windows 8.x and Chrome should auto-update their versions of Flash.

Windows users who browse the Web with anything other than Internet Explorer may need to apply this patch twice, once with IE and again using the alternative browser (Firefox, Opera, e.g.). If you have Adobe AIR installed (required by some programs like Pandora Desktop), you’ll want to update this program. AIR ships with an auto-update function that should prompt users to update when they start an application that requires it; the newest, patched version is v. 15 for Windows, Mac, and Android.

Adobe had also been scheduled to release updates today for Adobe Reader and Acrobat, but the company said it was pushing that release date back to the week of Sept. 15 to address some issues that popped up during testing of the patches.

As always, if you experience any issues updating these products, please leave a note about your troubles in the comments below.



PC-BSD at Fossetcon via Planet FreeBSD | 2014-09-09 15:13 UTC

Fossetcon will take place September 11–13 at the Rosen Plaza Hotel in Orlando, FL. Registration for this event ranges from $10 to $85 and includes meals and a t-shirt.

There will be a BSD booth in the expo area on both Friday and Saturday from 10:30–18:30. As usual, we’ll be giving out a bunch of cool swag, PC-BSD DVDs, and FreeNAS CDs, as well as accepting donations for the FreeBSD Foundation.  Dru Lavigne will present “ZFS 101″ at 11:30 on Saturday. The BSDA certification exam will be held at 15:00 On Saturday.



Nearly a week after this blog first reported signs that Home Depot was battling a major security incident, the company has acknowledged that it suffered a credit and debit card breach involving its U.S. and Canadian stores dating back to April 2014. Home Depot was quick to assure customers and banks that no debit card PIN data was compromised in the break-in. Nevertheless, multiple financial institutions contacted by this publication are reporting a steep increase over the past few days in fraudulent ATM withdrawals on customer accounts.

pwnddepot

The card data for sale in the underground that was stolen from Home Depot shoppers allows thieves to create counterfeit copies of debit and credit cards that can be used to purchase merchandise in big box stores. But if the crooks who buy stolen debit cards also are able to change the PIN on those accounts, the fabricated debit cards can then be used to withdraw cash from ATMs.

Experts say the thieves who are perpetrating the debit card fraud are capitalizing on a glut of card information stolen from Home Depot customers and being sold in cybercrime shops online. Those same crooks also are taking advantage of weak authentication methods in the automated phone systems that many banks use to allow customers to reset the PINs on their cards.

Here’s the critical part: The card data stolen from Home Depot customers and now for sale on the crime shop Rescator[dot]cc includes both the information needed to fabricate counterfeit cards as well as the legitimate cardholder’s full name and the city, state and ZIP of the Home Depot store from which the card was stolen (presumably by malware installed on some part of the retailer’s network, and probably on each point-of-sale device).

This is especially helpful for fraudsters since most Home Depot transactions are likely to occur in the same or nearby ZIP code as the cardholder. The ZIP code data of the store is important because it allows the bad guys to quickly and more accurately locate the Social Security number and date of birth of cardholders using criminal services in the underground that sell this information.

Why do the thieves need Social Security and date of birth information? Countless banks in the United States let customers change their PINs with a simple telephone call, using an automated call-in system known as a Voice Response Unit (VRU). A large number of these VRU systems allow the caller to change their PIN provided they pass three out of five security checks. One is that the system checks to see if the call is coming from a phone number on file for that customer. It also requests the following four pieces of information:

-the 3-digit code (known as a card verification value or CVV/CV2) printed on the back of the debit card;
-the card’s expiration date;
-the customer’s date of birth;
-the last four digits of the customer’s Social Security number.

On Thursday, I spoke with a fraud fighter at a bank in New England that experienced more than $25,000 in PIN debit fraud at ATMs in Canada. The bank employee said thieves were able to change the PINs on the cards using the bank’s automated VRU system. In this attack, the fraudsters were calling from disposable, prepaid Magic Jack telephone numbers, and they did not have the Cv2 for each card. But they were able to supply the other three data points.

KrebsOnSecurity also heard from an employee at a much larger bank on the West Coast that lost more than $300,000 in two hours today to PIN fraud on multiple debit cards that had all been used recently at Home Depot. The manager said the bad guys called the customer service folks at the bank and provided the last four of each cardholder’s Social Security number, date of birth, and the expiration date on the card. And, as with the bank in New England, that was enough information for the bank to reset the customer’s PIN.

The fraud manager said the scammers in this case also told the customer service people they were traveling in Italy, which made two things possible: It raised the withdrawal limits on the debit cards and allowed thieves to withdraw $300,000 in cash from Italian ATMs in the span of less than 120 minutes.

One way that banks can decrease the incidence of PIN reset fraud is to require that callers supply all of the requested information accurately, and indeed the bank employee I heard from in New England said a nearby financial institution she’d contacted that used the same VRU system saw its PIN fraud drop to zero when it began requiring that all questions be correctly answered. The bank on the West Coast that I interviewed also said it had already begun requiring all five elements before processing PIN changes on any cards that have been used at Home Depot since April.

Still, some of the world’s largest banks have begun moving away from so-called knowledge-based authentication for their VRU systems toward more robust technologies, such as voice biometrics and phone printing, said Avivah Litan, a fraud analyst with Gartner Inc.

“We saw this same activity in the wake of the breach at Target, where the thieves would call in and use the VRUs to check balances, remove blocks on cards, get the payment history and of course change PINs,” Litan said.

Voice biometric technologies create an index of voice fingerprints both for customers and various fraudsters who conduct VRU fraud, but Litan said fraudsters often will use voice synthesizers to defeat this layer of detection.

Phone printing profiles good and bad callers alike, building fingerprints based on dozens of call characteristics, including packet loss, dropped frames, noise, call clarity, phone type and a host of other far more geeky concepts (e.g., “quantization,” and “taggers“).

ANALYSIS

The fact that it is still possible to use customer service or an automated system to change someone else’s PIN with just the cardholder’s Social Security number, birthday and the expiration date of their stolen card is remarkable, and suggests that most banks remain clueless or willfully blind to the sophistication of identity theft services offered in the cybercrime underground. I know of at least two very popular and long-running cybercrime stores that sell this information for a few dollars apiece. One of them even advertises the sale of this information on more than 300 million Americans.

ssnfind copy

Banks are long overdue to move away from knowledge-based authentication. Forget about the fact that most major providers of these services have been shown to be compromised in the past year by the very crooks selling Social Security numbers and other data to identity thieves: The sad truth is that today’s cybercriminals are more likely to know the correct answers to these questions than you are.

I bring this up mainly because Home Depot is, predictably, offering credit monitoring services to affected customers (which, given the length of this breach is likely to impact a significant chunk of the American population). Credit and debit card fraud is annoying and inconvenient and can be at least temporarily expensive for victims, but as long as you are keeping a close eye on your monthly statements and reporting any unauthorized charges immediately, you will not be on the hook for those charges.

Please note that credit monitoring services will not help with this task, as they are not designed to look for fraud on existing accounts tied to your name and personal information. As I’ve noted in several stories, credit monitoring services are of dubious value because although they may alert you when thieves open new lines of credit in your name, those services do not prevent that activity. The one thing these services are good for is in helping identity theft victims clean up the mess and repair their good name.

However, given the fact that your Social Security number, date of birth and every possible answer to all of these knowledge-based authentication questions can be had for $25 in order to establish new lines of credit in your name, it makes good sense for people to avail themselves of free credit monitoring services. But there is little reason to pay for these services. If you don’t already have a credit monitoring service for free then maybe you haven’t been paying close enough attention to the dozens of companies over the past year that have likely lost your data in a breach and are already offering these services for free.

For more information about the benefits and limits of credit monitoring services — as well as other helpful tips to proactively safeguard your credit file — see this story.

More information, including an FAQ about the breach, released by Home Depot is available at this link.





Gentoo News

Council News

Concerning the handling of bash-completion and of phase functions in eclasses in general the council decided no actions. The former should be handled by the shell-tools team, the latter needs more discussion on the mailing lists.

Then we had two hot topics. The first was the games team policy; the council clarified that the games team has in no way authority over game ebuilds maintained by other developers. In addition, the games team should elect a lead in the near future. If it doesn’t it will be considered dysfunctional.  Tim Harder (radhermit) acts as interim lead and organizes the elections.

Next, rumors about the handling of dynamic dependencies in Portage had sparked quite a stir. The council asks the Portage team basically not to remove dynamic dependency handling before they haven’t worked out and presented a good plan how Gentoo would work without them. Portage tree policies and the
handling of eclasses and virtuals in particular need to be clarified.

Finally the list of planned features for EAPI 6 was amended by two items, namely additional options for configure and a non-runtime switchable ||= () or-dependency.

Gentoo Developer Moves

Summary

Gentoo is made up of 242 active developers, of which 43 are currently away.
Gentoo has recruited a total of 803 developers since its inception.

Changes

  • Ian Stakenvicius (axs) joined the multilib project
  • Michał Górny (mgorny) joined the QA team
  • Kristian Fiskerstrand (k_f) joined the Security team
  • Richard Freeman (rich0) joined the systemd team
  • Pavlos Ratis (dastergon) joined the Gentoo Infrastructure team
  • Patrice Clement (monsieur) and Ian Stakenvicius (axs) joined the perl team
  • Chris Reffett (creffett) joined the Wiki team
  • Pavlos Ratis (dastergon) left the KDE project
  • Dirkjan Ochtman (djc) left the ComRel project

Portage

This section summarizes the current state of the portage tree.

Architectures 45
Categories 162
Packages 17653
Ebuilds 37397
Architecture Stable Testing Total % of Packages
alpha 3661 574 4235 23.99%
amd64 10895 6263 17158 97.20%
amd64-fbsd 0 1573 1573 8.91%
arm 2692 1755 4447 25.19%
arm64 570 32 602 3.41%
hppa 3073 496 3569 20.22%
ia64 3196 626 3822 21.65%
m68k 614 98 712 4.03%
mips 0 2410 2410 13.65%
ppc 6841 2475 9316 52.77%
ppc64 4332 971 5303 30.04%
s390 1464 349 1813 10.27%
sh 1650 427 2077 11.77%
sparc 4135 922 5057 28.65%
sparc-fbsd 0 317 317 1.80%
x86 11572 5297 16869 95.56%
x86-fbsd 0 3241 3241 18.36%

gmn-portage-stats-2014-09

Security

The following GLSAs have been released by the Security Team

GLSA Package Description Bug
201408-19 app-office/openoffice-bin (and 3 more) OpenOffice, LibreOffice: Multiple vulnerabilities 283370
201408-18 net-analyzer/nrpe NRPE: Multiple Vulnerabilities 397603
201408-17 app-emulation/qemu QEMU: Multiple vulnerabilities 486352
201408-16 www-client/chromium Chromium: Multiple vulnerabilities 504328
201408-15 dev-db/postgresql-server PostgreSQL: Multiple vulnerabilities 456080
201408-14 net-misc/stunnel stunnel: Information disclosure 503506
201408-13 dev-python/jinja Jinja2: Multiple vulnerabilities 497690
201408-12 www-servers/apache Apache HTTP Server: Multiple vulnerabilities 504990
201408-11 dev-lang/php PHP: Multiple vulnerabilities 459904
201408-10 dev-libs/libgcrypt Libgcrypt: Side-channel attack 519396
201408-09 dev-libs/libtasn1 GNU Libtasn1: Multiple vulnerabilities 511536
201408-08 sys-apps/file file: Denial of Service 505534
201408-07 media-libs/libmodplug ModPlug XMMS Plugin: Multiple vulnerabilities 480388
201408-06 media-libs/libpng libpng: Multiple vulnerabilities 503014
201408-05 www-plugins/adobe-flash Adobe Flash Player: Multiple vulnerabilities 519790
201408-04 dev-util/catfish Catfish: Multiple Vulnerabilities 502536
201408-03 net-libs/libssh LibSSH: Information disclosure 503504
201408-02 media-libs/freetype FreeType: Arbitrary code execution 504088
201408-01 dev-php/ZendFramework Zend Framework: SQL injection 369139

Package Removals/Additions

Removals

Package Developer Date
virtual/perl-Class-ISA dilfridge 02 Aug 2014
virtual/perl-Filter dilfridge 02 Aug 2014
dev-vcs/gitosis robbat2 04 Aug 2014
dev-vcs/gitosis-gentoo robbat2 04 Aug 2014
virtual/python-argparse mgorny 11 Aug 2014
virtual/python-unittest2 mgorny 11 Aug 2014
app-emacs/sawfish ulm 19 Aug 2014
virtual/ruby-test-unit graaff 20 Aug 2014
games-action/d2x mr_bones_ 25 Aug 2014
games-arcade/koules mr_bones_ 25 Aug 2014
dev-lang/libcilkrts ottxor 26 Aug 2014

Additions

Package Developer Date
dev-python/oslotest prometheanfire 01 Aug 2014
dev-db/tokumx chainsaw 01 Aug 2014
sys-boot/gummiboot mgorny 02 Aug 2014
app-admin/supernova alunduil 03 Aug 2014
dev-db/mysql-cluster robbat2 03 Aug 2014
net-libs/txtorcon mrueg 04 Aug 2014
dev-ruby/prawn-table mrueg 06 Aug 2014
sys-apps/cv zx2c4 06 Aug 2014
media-libs/openctm amynka 07 Aug 2014
sci-libs/levmar amynka 07 Aug 2014
media-gfx/printrun amynka 07 Aug 2014
dev-python/alabaster idella4 10 Aug 2014
dev-haskell/regex-pcre slyfox 11 Aug 2014
dev-python/gcs-oauth2-boto-plugin vapier 12 Aug 2014
dev-python/astropy-helpers jlec 12 Aug 2014
dev-perl/Math-ModInt chainsaw 13 Aug 2014
dev-ruby/classifier-reborn mrueg 13 Aug 2014
media-gfx/meshlab amynka 14 Aug 2014
dev-libs/librevenge scarabeus 15 Aug 2014
www-apps/jekyll-coffeescript mrueg 15 Aug 2014
www-apps/jekyll-gist mrueg 15 Aug 2014
www-apps/jekyll-paginate mrueg 15 Aug 2014
www-apps/jekyll-watch mrueg 15 Aug 2014
sec-policy/selinux-salt swift 15 Aug 2014
www-apps/jekyll-sass-converter mrueg 15 Aug 2014
dev-ruby/rouge mrueg 15 Aug 2014
dev-ruby/ruby-beautify graaff 16 Aug 2014
sys-firmware/nvidia-firmware idl0r 17 Aug 2014
media-libs/libmpris2client ssuominen 20 Aug 2014
xfce-extra/xfdashboard ssuominen 20 Aug 2014
www-client/opera-developer jer 20 Aug 2014
dev-libs/openspecfun patrick 21 Aug 2014
dev-libs/marisa dlan 22 Aug 2014
media-sound/dcaenc beandog 22 Aug 2014
sci-mathematics/geogebra amynka 23 Aug 2014
dev-python/crumbs alunduil 25 Aug 2014
media-gfx/kxstitch kensington 26 Aug 2014
media-gfx/symboleditor kensington 26 Aug 2014
dev-perl/Sort-Key chainsaw 26 Aug 2014
dev-perl/Sort-Key-IPv4 chainsaw 26 Aug 2014
sci-visualization/yt xarthisius 26 Aug 2014
dev-ruby/globalid graaff 27 Aug 2014
dev-python/certifi idella4 27 Aug 2014
www-apps/jekyll-sitemap mrueg 27 Aug 2014
sys-apps/tuned dlan 29 Aug 2014
app-portage/g-sorcery jauhien 29 Aug 2014
app-portage/gs-elpa jauhien 29 Aug 2014
app-portage/gs-pypi jauhien 29 Aug 2014
app-admin/eselect-rust jauhien 29 Aug 2014
sys-block/raid-check chutzpah 29 Aug 2014
dev-python/python3-openid maksbotan 30 Aug 2014
dev-python/python-social-auth maksbotan 30 Aug 2014
dev-python/websocket-client alunduil 31 Aug 2014
dev-ruby/ethon graaff 31 Aug 2014

Bugzilla

The Gentoo community uses Bugzilla to record and track bugs, notifications, suggestions and other interactions with the development team.

Activity

The following tables and charts summarize the activity on Bugzilla between 01 August 2014 and 31 August 2014. Not fixed means bugs that were resolved as NEEDINFO, WONTFIX, CANTFIX, INVALID or UPSTREAM.
gmn-activity-2014-08

Bug Activity Number
New 1575
Closed 981
Not fixed 187
Duplicates 145
Total 6023
Blocker 5
Critical 19
Major 66

Closed bug ranking

The following table outlines the teams and developers with the most bugs resolved during this period

Rank Team/Developer Bug Count
1 Gentoo Security 102
2 Gentoo's Team for Core System packages 39
3 Gentoo KDE team 37
4 Default Assignee for Orphaned Packages 32
5 Julian Ospald (hasufell) 26
6 Gentoo Games 25
7 Portage team 25
8 Netmon Herd 24
9 Python Gentoo Team 23
10 Others 647

gmn-closed-2014-08

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Linux bug wranglers 160
2 Gentoo Security 61
3 Default Assignee for Orphaned Packages 60
4 Gentoo KDE team 45
5 Gentoo's Team for Core System packages 45
6 Gentoo Linux Gnome Desktop Team 37
7 Gentoo Games 28
8 Portage team 28
9 Python Gentoo Team 26
10 Others 1084

gmn-opened-2014-08

Heard in the community

Send us your favorite Gentoo script or tip at gmn@gentoo.org

Getting Involved?

Interested in helping out? The GMN relies on volunteers and members of the community for content every month. If you are interested in writing for the GMN or thinking of another way to contribute, please send an e-mail to gmn@gentoo.org.

Comments or Suggestions?

Please head over to this forum post.



The PC-BSD team is pleased to announce the availability of the next PC-BSD quarterly package update, version 10.0.3!

This update includes a number of important bug-fixes, as well as newer packages and desktops. Packages such as Chromium 37.0.2062.94, Cinnamon 2.2.14, Lumina 0.6.2 and more. This release also includes a CD-sized ISO of TrueOS, for users who want to install a server without X. For more details and updating instructions, refer to the notes below.

We are already hard at work on the next major release of PC-BSD, 10.1 later this fall, which will include FreeBSD 10.1-RELEASE under the hood. Users interested in following along with development should sign up for our Testing mailing list.

PC-BSD Notable Changes

* Cinnamon 2.2.14
* Chromium 37.0.2062.94
* NVIDIA Driver 340.24
* Lumina desktop 0.6.2-beta
* Pkg 1.3.7
* Various fixes to the Appcafe Qt UI
* Bugfixes to Warden / jail creation
* Fixed a bug with USB media not always being bootable
* Fixed several issues with Xorg setup
* Improved Boot-Environments to allow “beadm activate” to set default
* Support for jail “bulk” creation via Warden
* Fixes for relative ZFS dataset mount-point creation via Warden
* Support for full-disk (GELI) encryption without an unencrypted /boot partition

TrueOS

Along with our traditional PC-BSD DVD ISO image, we have also created a CD-sized ISO image of TrueOS, our server edition.

This is a text-based installer which includes FreeBSD 10.0-Release under the hood. It includes the following features:

* ZFS on Root installation
* Boot-Environment support
* Command-Line versions of PC-BSD utilities, such as Warden, Life-Preserver and more.
* Support for full-disk (GELI) encryption without an unencrypted /boot partition

We have some additional features also in the works for 10.1 and later, stay tuned this fall for more information.

Updating

Due to some changes with how pkgng works, it is recommended that all users update via the command-line using the following steps:

# pkg update –f
# pkg upgrade pkg
# pkg update –f
# pkg upgrade
# pc-extractoverlay ports
# reboot

PKGNG may need to re-install many of your packages to fix an issue with shared library version detection. If you run into issues doing this, or have conflicts, please open a bug report with the output of the above commands.

If you run into shared library issues running programs after upgrading, you may need to do a full-upgrade with the following:

# pkg upgrade –f

Getting media

10.0.3 DVD/USB media can be downloaded from this URL via HTTP or Torrent.

Reporting Bugs
Found a bug in 10.0.3? Please report it (with as much detail as possible) to our new RedMine Database.



The apparent credit and debit card breach uncovered last week at Home Depot was aided in part by a new variant of the malicious software program that stole card account data from cash registers at Target last December, according to sources close to the investigation.

Photo: Nicholas Eckhart

Photo: Nicholas Eckhart

On Tuesday, KrebsOnSecurity broke the news that Home Depot was working with law enforcement to investigate “unusual activity” after multiple banks said they’d traced a pattern of card fraud back to debit and credit cards that had all been used at Home Depot locations since May of this year.

A source close to the investigation told this author that an analysis revealed at least some of Home Depot’s store registers had been infected with a new variant of “BlackPOS” (a.k.a. “Kaptoxa”), a malware strain designed to siphon data from cards when they are swiped at infected point-of-sale systems running Microsoft Windows.

The information on the malware adds another indicator that those responsible for the as-yet unconfirmed breach at Home Depot also were involved in the December 2013 attack on Target that exposed 40 million customer debit and credit card accounts. BlackPOS also was found on point-of-sale systems at Target last year. What’s more, cards apparently stolen from Home Depot shoppers first turned up for sale on Rescator[dot]cc, the same underground cybercrime shop that sold millions of cards stolen in the Target attack.

Clues buried within this newer version of BlackPOS support the theory put forth by multiple banks that the Home Depot breach may involve compromised store transactions going back at least several months. In addition, the cybercrime shop Rescator over the past few days pushed out nine more large batches of stolen cards onto his shop, all under the same “American Sanctions” label assigned to the first two batches of cards that originally tipped off banks to a pattern of card fraud that traced back to Home Depot. Likewise, the cards lifted from Target were sold in several dozen batches released over a period of three months on Rescator’s shop.

The cybercrime shop Rescator[dot]cc pushed out nine new batches of cards from the same

The cybercrime shop Rescator[dot]cc pushed out nine new batches of cards from the same “American Sanctions” base of cards that banks traced back to Home Depot.

POWERFUL ENEMIES

The tip from a source about BlackPOS infections found at Home Depot comes amid reports from several security firms about the discovery of a new version of BlackPOS. On Aug. 29, Trend Micro published a blog post stating that it had identified a brand new variant of BlackPOS in the wild that was targeting retail accounts. Trend said the updated version, which it first spotted on Aug. 22, sports a few notable new features, including an enhanced capability to capture card data from the physical memory of infected point-of-sale devices. Trend said the new version also has a feature that disguises the malware as a component of the antivirus product running on the system.

Contents of the new BlackPOS component responsible for exfiltrating stolen cards from the network. Source: Trend Micro.

Contents of the new BlackPOS component responsible for exfiltrating stolen cards from the network. Source: Trend Micro.

Trend notes that the new BlackPOS variant uses a similar method to offload stolen card data as the version used in the attack on Target.

“In one the biggest data breach[es] we’ve seen in 2013, the cybercriminals behind it offloaded the gathered data to a compromised server first while a different malware running on the compromised server uploaded it to the FTP,” wrote Trend’s Rhena Inocencio. “We surmise that this new BlackPOS malware uses the same exfiltration tactic.”

An Internet search on the unique malware “hash” signature noted in Trend’s malware writeup indicates that the new BlackPOS verison was created on June 22, 2014, and that as late as Aug. 15, 2014 only one of more than two-dozen anti-malware tools (McAfee) detected it as malicious.

ANTI-AMERICAN MALWARE

Other clues in the new BlackPOS malware variant further suggest a link between the cybercrooks behind the apparent breach at Home Depot and the hackers who hit Target. The new BlackPOS variant includes several interesting text strings. Among those are five links to Web sites featuring content about America’s role in foreign conflicts, particularly in Libya and Ukraine.

One of the images linked to in the guts of the BlackPOS code.

One of the images linked to in the guts of the BlackPOS code.

Three of the links point to news, editorial articles and cartoons that accuse the United States of fomenting war and unrest in the name of Democracy in Ukraine, Syria, Egypt and Libya. One of the images shows four Molotov cocktails with the flags of those four nations on the bottles, next to a box of matches festooned with the American flag and match ready to strike. Another link leads to an image of the current armed conflict in Ukraine between Ukrainian forces and pro-Russian separatists.

This is interesting given what we know about Rescator, the individual principally responsible for running the store that is selling all of these stolen credit and debit cards. In the wake of the Target breach, I traced a long list of clues from Rescator’s various online identities back to a young programmer in Odessa, Ukraine. In his many personas, Rescator identified himself as a member of the Lampeduza cybercrime forum, and indeed this site is where he alerts customers about new batches of stolen cards.

As I discovered in my profile of Rescator, he and his crew seemed somewhat taken with the late despotic Libyan leader Muammar Gaddafi, although they prefer the phonetic spelling of his name. The Web site kaddafi[dot]hk was among four main carding shops run by Rescator’s crew (it has since been retired and merged with Rescator[dot]cc). The domain kaddafi[dot]me was set up to serve as an instant message Jabber server for cybercrooks, advertising its lack of logging and record keeping as a reason crooks should trust kaddafi[dot]me to handle their private online communications.

When I reached out to Rescator last December to obtain comment about my findings on his apparent role in the Target break-in, I received an instant message reply from the Jabber address “kaddafi@kaddafi[dot]me” (in that conversation, the person chatting with me from that address offered to pay me $10,000 if I did not run that story; I declined). But I also discovered that the kaddafi[dot]me domain was a blog of sorts that hosted some harsh and frankly chilling anti-American propaganda.

The entire three-part manifesto posted on the kaddafi[dot]me home page is no longer available, but a professionally translated snippet of this tirade reads:

“The movement of our Republic, the ideology of Lampeduza – is the opposition to Western countries, primarily targeting the restoration of the balance of forces in the world. After the collapse of the USSR, we have lost this fragile equilibrium face of the planet. We – the Senate and the top people of the Republic are not just fighting for survival and our place under the sun, we are driven by the idea! The idea, which is ​​living in all of us – to return all that was stolen and taken from our friendly countries grain by grain! We are fighting for a good cause! Hot blood is flowing in us, in citizens, who want to change situation in the world. We do not bend to other people’s opinions and desires, and give an adequate response to the Western globalism. It is essential to be a fighter for justice!

Perhaps we would be living completely differently now, if there had not been the plan of Allen Dulles, and if America had not invested billions in the collapse of the USSR. We were deprived of a common homeland, but not deprived of unity, have found our borders, and are even closer to each other. We saw the obvious principles of capitalism, where man to a man is a wolf [[see here for more context on this metaphor]]. Together, we can do a lot to bring back all the things that we have been deprived of because of America! We will be heard!

Citizens of Lampeduza – “free painters” ready to create and live the idea for the good of the Motherland — let’s first bend them over, and then insert deeper!!!

Google-translated version of Kaddafi[dot]me homepage.

Google-translated version of Kaddafi[dot]me homepage.



The Foundation recently sponsored Damian Vicino to attend BSDDay Argentina. Here is his trip report:

BSDday is the only BSD conference in South America as far as I know. The event's inception was in 2008 by 2 BSD Users Groups in Buenos Aires City. participated as part of the organisation committee from 2009 - 2012. In 2013, the event had no edition because of some big changes in the livesof the people participating in the organisation committee. In my case, I moved out of the country (and the continent). Thanks to the FreeBSD Foundation, I was able to return to South America for a few weeks this year to re-float the committee and the event, making possible the run of a 5th edition.

We started the preparation a few months before by coordinating remotely, but there was a lot of stuff to be done in-place, so I traveled 10 days earlier. In the days before the event, I coordinated with Universidad de Buenos Aires to finish the arrangements for the space to run the event and the supplies needed for the event. I worked as the main contact for the university and dealt with all the paperwork; being the largest university in Argentina, there is a lot of paperwork for everything. An interesting institutional plus this year is that the Faculty of Science and Department of Computer Science of Universidad de Buenos Aires declared officially the BSDday as an Event of Interest. Simultaneously, Hernan Constante and Matias Celani were coordinating accommodations for one of the speakers who traveled from Mar del Plata and making arrangements to have food & coffee for the event. Thanks for their help and also to Alejandro Lazaro who was helping in all he could remotely since he also moved out of Buenos Aires.

The quantity of proposals for talks received this year was about half the usual. We contacted previous speakers for feedback and we decided to include discussion spaces to find out why and how we can make it better for next year. On August 9th, a few minutes before the event started, the first speaker had family emergency. We decided to delay the opening talk and use the time for a first open discussion about the event and its future. The attendance was the lowest ever, so we focused the first discussion space on this topic. It appears to be a consensus that August is not a good month for the conference, because of the power outages in Buenos Aires in summer. From previous years, we knew that November is not good either. Another apparent reason is the break in continuity of the event (in 2013). Everyone in the room actively participated in the open discussion spaces. We noticed from discussions that the demographics of the event had changed. This time, we had a group of desktop users, mostly from FreeBSD, while in previous years we had mostly sysadmins from OpenBSD working in large companies or ISPs.

After the discussion, I did the opening talk with the help of Hernan Constante. The talk was also open to discussion so it extended a little longer than programmed; lucky for us, having only 1 track, it didn't affect the schedule much. The second talk was for 40 minutes, but was extended up to 2 hours and ended up in a different topic than the one it started with. We were tempted to stop it, but people were asking so many questions that we let it flow. We then had 4 more talks (including mine) and 2 more spaces for open discussion about anything-BSD where we collected opinions about the event, about BSD in Argentina, and the future of BSD advocacy actions. Since we didn't have sponsors for the food/coffee/supplies, we asked if anyone wanted to contribute at the end of the event. We were glad to see that everyone in the room put in money and we almost covered every expense for the event in this way. After the event, about 90% of the people moved to the bar across the street to share some beers and we kept discussing until the bar closed and kicked us out.

The week after the event, I met again with some organisers to discuss ideas for next year and do some analysis of what happened this year. One week later, I met with some companies and professionals to check sponsoring possibilities for next year's edition.

Last week, I collected and processed the materials we obtained from the event: videos, photos, and slides from every presentation. I still need to recover a few videos that we had to download to one of the organiser's computer (who left the country before me). In the following weeks, we will upload the videos, slides and pictures and formally close this year's event in order to start working for the 6th edition, expected to happen in 2016.

Once again, thank you very much to the FreeBSD Foundation for helping me with the expenses for this trip, to the University of Buenos Aires Faculty of Science and Computer Science department for giving us the space and support, to Hernan Constante, Alejandro Lazaro, Matias Celani, the speakers, and all those who helped to make this event possible once again.


New data gathered from the cybercrime underground suggests that the apparent credit and debit card breach at Home Depot involves nearly all of the company’s stores across the nation.

Evidence that a major U.S. retailer had been hacked and was leaking card data first surfaced Tuesday on the cybercrime store rescator[dot]cc, the shop that was principally responsible for selling cards stolen in the Target, Sally Beauty, P.F. Chang’s and Harbor Freight credit card breaches.

As with cards put up for sale in the wake of those breaches, Rescator’s shop lists each card according to the city, state and ZIP code of the store from which each card was stolen. See this story for examples of this dynamic in the case of Sally Beauty, and this piece that features the same analysis on the stolen card data from the Target breach.

Stolen credit cards for sale on Rescator's site index each card by the city, state and ZIP of the retail store from which each card was stolen.

Stolen credit cards for sale on Rescator’s site index each card by the city, state and ZIP of the retail store from which each card was stolen.

The ZIP code data allows crooks who buy these cards to create counterfeit copies of the credit and debit cards, and use them to buy gift cards and high-priced merchandise from big box retail stores. This information is extremely valuable to the crooks who are purchasing the stolen cards, for one simple reason: Banks will often block in-store card transactions on purchases that occur outside of the legitimate cardholder’s geographic region (particularly in the wake of a major breach).

Thus, experienced crooks prefer to purchase cards that were stolen from stores near them, because they know that using the cards for fraudulent purchases in the same geographic area as the legitimate cardholder is less likely to trigger alerts about suspicious transactions — alerts that could render the stolen card data worthless for the thieves.

This morning, KrebsOnSecurity pulled down all of the unique ZIP codes in the card data currently for sale from the two batches of cards that at least four banks have now mapped back to previous transactions at Home Depot. KrebsOnSecurity also obtained a commercial marketing list showing the location and ZIP code of every Home Depot store across the country.

Here’s the kicker: A comparison of the ZIP code data between the unique ZIPs represented on Rescator’s site, and those of the Home Depot stores shows a staggering 99.4 percent overlap.

Home Depot has not yet said for certain whether it has in fact experienced a store-wide card breach; rather, the most that the company is saying so far is that it is investigating “unusual activity” and that it is working with law enforcement on an investigation. Here is the page that Home Depot has set up for further notices about this investigation.

I double checked the data with several sources, including with Nicholas Weaver, a researcher at the International Computer Science Institute (ICSI) and at the University California, Berkeley. Weaver said the data suggests a very strong correlation.

“A 99+ percent overlap in ZIP codes strongly suggests that this source is from Home Depot,” Weaver said.

Here is a list of all unique ZIP codes represented in more than 3,000 debit and credit cards currently for sale on Rescator’s site (Rescator limits the number of cards one can view to the first 33 pages of results, 50 cards per page). Here is a list of all unique Home Depot ZIP codes, in case anyone wants to double check my work.

In all, there were 1,822 ZIP codes represented in the card data for sale on Rescator’s site, and 1,939 unique ZIPs corresponding to Home Depot store locations (while Home Depot says it has ~2,200 stores, it is safe to assume that some ZIP codes have more than one Home Depot store). Between those two lists of ZIP codes, there are 10 ZIP codes in Rescator’s card data that do not correspond to actual Home Depot stores.

Finally, there were 127 ZIP codes for Home Depot stores that were not in the list of ZIPs represented in Rescator’s card data. However, it’s important to note that the data pulled from Rescator’s site is almost certainly a tiny fraction of the cards that his shop will put up for sale in the coming days and weeks.

What does all this mean? Well, assuming Home Depot does confirm a breach, it could give us one way to determine the likely size of this breach. The banks I spoke with in reporting this story say the data they’re looking at suggests that the breach probably started in late April or early May. To put that in perspective, the Target breach impacted just shy of 1,800 stores, lasted for approximately three weeks, and resulted in the theft of roughly 40 million debit and credit card numbers. If a breach at Home Depot is confirmed, and if this analysis is correct, this breach could be much, much bigger than Target.

How does this affect you, dear reader? It’s important for Americans to remember that you have zero fraud liability on your credit card. If the card is compromised in a data breach and fraud occurs, any fraudulent charges will be reversed. BUT, not all fraudulent charges may be detected by the bank that issued your card, so it’s important to monitor your account for any unauthorized transactions and report those bogus charges immediately.



Ever since October 2013, when the FBI took down the online black market and drug bazaar known as the Silk Road, privacy activists and security experts have traded conspiracy theories about how the U.S. government managed to discover the geographic location of the Silk Road Web servers. Those systems were supposed to be obscured behind the anonymity service Tor, but as court documents released Friday explain, that wasn’t entirely true: Turns out, the login page for the Silk Road employed an anti-abuse CAPTCHA service that pulled content from the open Internet, thus leaking the site’s true location.

leakyshipTor helps users disguise their identity by bouncing their traffic between different Tor servers, and by encrypting that traffic at every hop along the way. The Silk Road, like many sites that host illicit activity, relied on a feature of Tor known as “hidden services.” This feature allows anyone to offer a Web server without revealing the true Internet address to the site’s users.

That is, if you do it correctly, which involves making sure you aren’t mixing content from the regular open Internet into the fabric of a site protected by Tor. But according to federal investigators,  Ross W. Ulbricht — a.k.a. the “Dread Pirate Roberts,” the 30-year-old arrested last year and charged with running the Silk Road — made this exact mistake.

As explained in the Tor how-to, in order for the Internet address of a computer to be fully hidden on Tor, the applications running on the computer must be properly configured for that purpose. Otherwise, the computer’s true Internet address may “leak” through the traffic sent from the computer.

howtorworks

And this is how the feds say they located the Silk Road servers:

“The IP address leak we discovered came from the Silk Road user login interface. Upon examining the individual packets of data being sent back from the website, we noticed that the headers of some of the packets reflected a certain IP address not associated with any known Tor node as the source of the packets. This IP address (the “Subject IP Address”) was the only non-Tor source IP address reflected in the traffic we examined.”

“The Subject IP Address caught our attention because, if a hidden service is properly configured to work on Tor, the source IP address of traffic sent from the hidden service should appear as the IP address of a Tor node, as opposed to the true IP address of the hidden service, which Tor is designed to conceal. When I typed the Subject IP Address into an ordinary (non-Tor) web browser, a part of the Silk Road login screen (the CAPTCHA prompt) appeared. Based on my training and experience, this indicated that the Subject IP Address was the IP address of the SR Server, and that it was ‘leaking’ from the SR Server because the computer code underlying the login interface was not properly configured at the time to work on Tor.”

For many Tor fans and advocates, The Dread Pirate Roberts’ goof will no doubt be labeled a noob mistake — and perhaps it was. But as I’ve said time and again, staying anonymous online is hard work, even for those of us who are relatively experienced at it. It’s so difficult, in fact, that even hardened cybercrooks eventually slip up in important and often fateful ways (that is, if someone or something was around at the time to keep a record of it).

A copy of the government’s declaration on how it located the Silk Road servers is here (PDF). A hat tip to Nicholas Weaver for the heads up about this filing.

A snapshop of offerings on the Silk Road.

A snapshop of offerings on the Silk Road.



Bash has many subtle pitfalls, some of them being able to live unnoticed for a very long time. A common example of that kind of pitfall is ubiquitous filename expansion, or globbing. What many script writers forget about to notice is that practically anything that looks like a pattern and is not quoted is subject to globbing, including unquoted variables.

There are two extra snags that add up to this. Firstly, many people forget that not only asterisks (*) and question marks (?) make up patterns — square brackets ([) do that as well. Secondly, by default bash (and POSIX shell) take failed expansions literally. That is, if your glob does not match any file, you may not even know that you are globbing.

It's all just a matter of running in the proper directory for the result to change. Of course, it's often unlikely — maybe even close to impossible. You can work towards preventing that by running in a safe directory. But in the end, writing predictable software is a fine quality.

How to notice mistakes?

Bash provides a two major facilities that could help you stop mistakes — shopts nullglob and failglob.

The nullglob option is a good choice for a default for your script. After enabling it, failing filename expansions result in no parameters rather than verbatim pattern itself. This has two important implications.

Firstly, it makes iterating over optional files easy:

for f in a/* b/* c/*; do
    some_magic "${f}"
done

Without nullglob, the above may actually return a/* if no file matches the pattern. For this reason, you would need to add an additional check for existence of file inside the loop. With nullglob, it will just ‘omit’ the unmatched arguments. In fact, if none of the patterns match the loop won't be run even once.

Secondly, it turns every accidental glob into null. While this isn't the most friendly warning and in fact it may have very undesired results, you're more likely to notice that something is going wrong.

The failglob option is better if you can assume you don't need to match files in its scope. In this case, bash treats every failing filename expansion as a fatal error and terminates execution with an appropriate message.

The main advantage of failglob is that it makes you aware of any mistake before someone hits it the hard way. Of course, assuming that it doesn't accidentally expand into something already.

There is also a choice of noglob. However, I wouldn't recommend it since it works around mistakes rather than fixing them, and makes the code rely on a non-standard environment.

Word splitting without globbing

One of the pitfalls I myself noticed lately is the attempt of using unquoted variable substitution to do word splitting. For example:

for i in ${v}; do
    echo "${i}"
done

At a first glance, everything looks fine. ${v} contains a whitespace-separated list of words and we iterate over each word. The pitfall here is that words in ${v} are subject to filename expansion. For example, if a lone asterisk would happen to be there (like v='10 * 4'), you'd actually get all files in the current directory. Unexpected, isn't it?

I am aware of three solutions that can be used to accomplish word splitting without implicit globbing:

  1. setting shopt -s noglob locally,
  2. setting GLOBIGNORE='*' locally,
  3. using the swiss army knife of read to perform word splitting.

Personally, I dislike the first two since they require set-and-restore magic, and the latter also has the penalty of doing the globbing then discarding the result. Therefore, I will expand on using read:

read -r -d '' -a words <<<"${v}"
for i in "${words[@]}"; do
    echo "${i}"
done

While normally read is used to read from files, we can use the here string syntax of bash to feed the variable into it. The -r option disables backslash escape processing that is undesired here. -d '' causes read to process the whole input and not stop at any delimiter (like newline). -a words causes it to put the split words into array ${words[@]} — and since we know how to safely iterate over an array, the underlying issue is solved.



32bit Madness via Planet Gentoo | 2014-09-05 06:41 UTC

This week I ran into a funny issue doing backups with rsync:
rsnapshot/weekly.3/server/storage/lost/of/subdirectories/some-stupid.file => rsnapshot/daily.0/server/storage/lost/of/subdirectories/some-stupid.file
ERROR: out of memory in make_file [generator]
rsync error: error allocating core memory buffers (code 22) at util.c(117) [generator=3.0.9]
rsync error: received SIGUSR1 (code 19) at main.c(1298) [receiver=3.0.9]
rsync: connection unexpectedly closed (2168136360 bytes received so far) [sender]
rsync error: error allocating core memory buffers (code 22) at io.c(605) [sender=3.0.9]
Oopsiedaisy, rsync ran out of memory. But ... this machine has 8GB RAM, plus 32GB Swap ?!
So I re-ran this and started observing, and BAM, it fails again. With ~4GB RAM free.

4GB you say, eh? That smells of ... 2^32 ...
For doing the copying I was using sysrescuecd, and then it became obvious to me: All binaries are of course 32bit!

So now I'm doing a horrible hack of "linux64 chroot /mnt/server" so that I have a 64bit environment that does not run out of space randomly. Plus 3 new bugs for the Gentoo livecd, which fails to appreciate USB and other things.
Who would have thought that a 16TB partition can make rsync stumble over address space limits ...



Figure 1.1: Iron Penguin

Fig. 1: Iron Penguin

Gentoo Linux is proud to announce the availability of a new LiveDVD to celebrate the continued collaboration between Gentoo users and developers, The LiveDVD features a superb list of packages, some of which are listed below.

A special thanks to the Gentoo Infrastructure Team and likewhoa. Their hard work behind the scenes provide the resources, services and technology necessary to support the Gentoo Linux project.

  • Packages included in this release: Linux Kernel 3.15.6, Xorg 1.16.0, KDE 4.13.3, Gnome 3.12.2, XFCE 4.10, Fluxbox 1.3.5, LXQT Desktop 0.7.0, i3 Desktop 2.8, Firefox 31.0, LibreOffice 4.2.5.2, Gimp 2.8.10-r1, Blender 2.71-r1, Amarok 2.8.0-r2, Chromium 37.0.2062.35 and much more ...
  • If you want to see if your package is included we have generated both the x86 package list, and amd64 package list. The FAQ is located at FAQ. DVD cases and covers for the 20140826 release are located at Artwork. Persistence mode is back in the 20140826 release!.

The LiveDVD is available in two flavors: a hybrid x86/x86_64 version, and an x86_64 multi lib version. The livedvd-x86-amd64-32ul-20140826 version will work on 32-bit x86 or 64-bit x86_64. If your CPU architecture is x86, then boot with the default gentoo kernel. If your arch is amd64, boot with the gentoo64 kernel. This means you can boot a 64-bit kernel and install a customized 64-bit user land while using the provided 32-bit user land. The livedvd-amd64-multilib-20140826 version is for x86_64 only.

If you are ready to check it out, let our bouncer direct you to the closest x86 image or amd64 image file.

If you need support or have any questions, please visit the discussion thread on our forum.

Thank you for your continued support,
Gentoo Linux Developers, the Gentoo Foundation, and the Gentoo-Ten Project.



By popular demand, the source tree for the Lumina project has just been moved to its own repository within the main PC-BSD project tree on GitHub.

In addition to this, an official FreeBSD port for Lumina was just committed to the FreeBSD ports tree which uses the new repo.

 

By the way, here is a quick usage summary for those that are interested in how “light” Lumina 0.6.2 is on PC-BSD 10.0.3:

System: Netbook with a single 1.6GHz atom processor and 2GB of memory (Fresh installation of PC-BSD 10.0.3 with Lumina 0.6.2)

Usage: ~0.2–0.4% CPU and ~120MB active memory use (no apps running except an xterm with “top” after a couple minutes for the PC-BSD tray applications to start up and settle down)

 



Multiple banks say they are seeing evidence that Home Depot stores may be the source of a massive new batch of stolen credit and debit cards that went on sale this morning in the cybercrime underground. Home Depot says that it is working with banks and law enforcement agencies to investigate reports of suspicious activity.

Contacted by this reporter about information shared from several financial institutions, Home Depot spokesperson Paula Drake confirmed that the company is investigating.

“I can confirm we are looking into some unusual activity and we are working with our banking partners and law enforcement to investigate,” Drake said, reading from a prepared statement. “Protecting our customers’ information is something we take extremely seriously, and we are aggressively gathering facts at this point while working to protect customers. If we confirm that a breach has occurred, we will make sure customers are notified immediately. Right now, for security reasons, it would be inappropriate for us to speculate further – but we will provide further information as soon as possible.”

There are signs that the perpetrators of this apparent breach may be the same group of Russian and Ukrainian hackers responsible for the data breaches at Target, Sally Beauty and P.F. Chang’s, among others. The banks contacted by this reporter all purchased their customers’ cards from the same underground store – rescator[dot]cc — which on Sept. 2 moved two massive new batches of stolen cards onto the market.

A massive new batch of cards labeled

A massive new batch of cards labeled “American Sanctions” and “European Sanctions” went on sale Tuesday, Sept. 2, 2014.

In what can only be interpreted as intended retribution for U.S. and European sanctions against Russia for its aggressive actions in Ukraine, this crime shop has named its newest batch of cards “American Sanctions.” Stolen cards issued by European banks that were used in compromised US store locations are being sold under a new batch of cards labled “European Sanctions.”

It is not clear at this time how many stores may have been impacted, but preliminary analysis indicates the breach may extend across all 2,200 Home Depot stores in the United States. Home Depot also operates some 287 stores outside the U.S. including in Canada, Guam, Mexico, and Puerto Rico.

This is likely to be a fast-moving story with several updates as more information becomes available. Stay tuned.

Update: 1:50 p.m. ET: Several banks contacted by this reporter said they believe this breach may extend back to late April or early May 2014. If that is accurate — and if even a majority of Home Depot stores were compromised — this breach could be many times larger than Target, which had 40 million credit and debit cards stolen over a three-week period.



PC-BSD 10.0.3-RC2 ISO images are now available for testing.

Users on the EDGE package set, or 10.0.3-RC1 can update to the newer set with the following commands:

# pkg update –f
# pkg upgrade
# pc-extractoverlay ports

This update brings in the newer pkgng 1.3.7, which may need to re-install many of your packages in order to properly fix an issue with shared-library version detection in previous pkgng releases.

The current plan is to release 10.0.3 early next week, so please let us know of any issues right away via our RedMine bug tracker.



AMD HSA via Planet Gentoo | 2014-09-03 06:25 UTC

With the release of the "Kaveri" APUs AMD has released some quite intriguing technology. The idea of the "APU" is a blend of CPU and GPU, what AMD calls "HSA" - Heterogenous System Architecture.
What does this mean for us? In theory, once software catches up, it'll be a lot easier to use GPU-acceleration (e.g. OpenCL) within normal applications.

One big advantage seems to be that CPU and GPU share the system memory, so with the right drivers you should be able to do zero-copy GPU processing. No more host-to-GPU copy and other waste of time.

So far there hasn't been any driver support to take advantage of that. Here's the good news: As of a week or two ago there is driver support. Still very alpha, but ... at last, drivers!

On the kernel side there's the kfd driver, which piggybacks on radeon. It's available in a slightly very patched kernel from AMD. During bootup it looks like this:
[    1.651992] [drm] radeon kernel modesetting enabled.
[    1.657248] kfd kfd: Initialized module
[    1.657254] Found CRAT image with size=1440
[    1.657257] Parsing CRAT table with 1 nodes
[    1.657258] Found CU entry in CRAT table with proximity_domain=0 caps=0
[    1.657260] CU CPU: cores=4 id_base=16
[    1.657261] Found CU entry in CRAT table with proximity_domain=0 caps=0
[    1.657262] CU GPU: simds=32 id_base=-2147483648
[    1.657263] Found memory entry in CRAT table with proximity_domain=0
[    1.657264] Found memory entry in CRAT table with proximity_domain=0
[    1.657265] Found memory entry in CRAT table with proximity_domain=0
[    1.657266] Found memory entry in CRAT table with proximity_domain=0
[    1.657267] Found cache entry in CRAT table with processor_id=16
[    1.657268] Found cache entry in CRAT table with processor_id=16
[    1.657269] Found cache entry in CRAT table with processor_id=16
[    1.657270] Found cache entry in CRAT table with processor_id=17
[    1.657271] Found cache entry in CRAT table with processor_id=18
[    1.657272] Found cache entry in CRAT table with processor_id=18
[    1.657273] Found cache entry in CRAT table with processor_id=18
[    1.657274] Found cache entry in CRAT table with processor_id=19
[    1.657274] Found TLB entry in CRAT table (not processing)
[    1.657275] Found TLB entry in CRAT table (not processing)
[    1.657276] Found TLB entry in CRAT table (not processing)
[    1.657276] Found TLB entry in CRAT table (not processing)
[    1.657277] Found TLB entry in CRAT table (not processing)
[    1.657278] Found TLB entry in CRAT table (not processing)
[    1.657278] Found TLB entry in CRAT table (not processing)
[    1.657279] Found TLB entry in CRAT table (not processing)
[    1.657279] Found TLB entry in CRAT table (not processing)
[    1.657280] Found TLB entry in CRAT table (not processing)
[    1.657286] Creating topology SYSFS entries
[    1.657316] Finished initializing topology ret=0
[    1.663173] [drm] initializing kernel modesetting (KAVERI 0x1002:0x1313 0x1002:0x0123).
[    1.663204] [drm] register mmio base: 0xFEB00000
[    1.663206] [drm] register mmio size: 262144
[    1.663210] [drm] doorbell mmio base: 0xD0000000
[    1.663211] [drm] doorbell mmio size: 8388608
[    1.663280] ATOM BIOS: 113
[    1.663357] radeon 0000:00:01.0: VRAM: 1024M 0x0000000000000000 - 0x000000003FFFFFFF (1024M used)
[    1.663359] radeon 0000:00:01.0: GTT: 1024M 0x0000000040000000 - 0x000000007FFFFFFF
[    1.663360] [drm] Detected VRAM RAM=1024M, BAR=256M
[    1.663361] [drm] RAM width 128bits DDR
[    1.663471] [TTM] Zone  kernel: Available graphics memory: 7671900 kiB
[    1.663472] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
[    1.663473] [TTM] Initializing pool allocator
[    1.663477] [TTM] Initializing DMA pool allocator
[    1.663496] [drm] radeon: 1024M of VRAM memory ready
[    1.663497] [drm] radeon: 1024M of GTT memory ready.
[    1.663516] [drm] Loading KAVERI Microcode
[    1.667303] [drm] Internal thermal controller without fan control
[    1.668401] [drm] radeon: dpm initialized
[    1.669403] [drm] GART: num cpu pages 262144, num gpu pages 262144
[    1.685757] [drm] PCIE GART of 1024M enabled (table at 0x0000000000277000).
[    1.685894] radeon 0000:00:01.0: WB enabled
[    1.685905] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000040000c00 and cpu addr 0xffff880429c5bc00
[    1.685908] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000040000c04 and cpu addr 0xffff880429c5bc04
[    1.685910] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000040000c08 and cpu addr 0xffff880429c5bc08
[    1.685912] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000040000c0c and cpu addr 0xffff880429c5bc0c
[    1.685914] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000040000c10 and cpu addr 0xffff880429c5bc10
[    1.686373] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000076c98 and cpu addr 0xffffc90012236c98
[    1.686375] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    1.686376] [drm] Driver supports precise vblank timestamp query.
[    1.686406] radeon 0000:00:01.0: irq 83 for MSI/MSI-X
[    1.686418] radeon 0000:00:01.0: radeon: using MSI.
[    1.686441] [drm] radeon: irq initialized.
[    1.689611] [drm] ring test on 0 succeeded in 3 usecs
[    1.689699] [drm] ring test on 1 succeeded in 2 usecs
[    1.689712] [drm] ring test on 2 succeeded in 2 usecs
[    1.689849] [drm] ring test on 3 succeeded in 2 usecs
[    1.689856] [drm] ring test on 4 succeeded in 2 usecs
[    1.711523] tsc: Refined TSC clocksource calibration: 3393.828 MHz
[    1.746010] [drm] ring test on 5 succeeded in 1 usecs
[    1.766115] [drm] UVD initialized successfully.
[    1.767829] [drm] ib test on ring 0 succeeded in 0 usecs
[    2.268252] [drm] ib test on ring 1 succeeded in 0 usecs
[    2.712891] Switched to clocksource tsc
[    2.768698] [drm] ib test on ring 2 succeeded in 0 usecs
[    2.768819] [drm] ib test on ring 3 succeeded in 0 usecs
[    2.768870] [drm] ib test on ring 4 succeeded in 0 usecs
[    2.791599] [drm] ib test on ring 5 succeeded
[    2.812675] [drm] Radeon Display Connectors
[    2.812677] [drm] Connector 0:
[    2.812679] [drm]   DVI-D-1
[    2.812680] [drm]   HPD3
[    2.812682] [drm]   DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[    2.812683] [drm]   Encoders:
[    2.812684] [drm]     DFP2: INTERNAL_UNIPHY2
[    2.812685] [drm] Connector 1:
[    2.812686] [drm]   HDMI-A-1
[    2.812687] [drm]   HPD1
[    2.812688] [drm]   DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[    2.812689] [drm]   Encoders:
[    2.812690] [drm]     DFP1: INTERNAL_UNIPHY
[    2.812691] [drm] Connector 2:
[    2.812692] [drm]   VGA-1
[    2.812693] [drm]   HPD2
[    2.812695] [drm]   DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[    2.812695] [drm]   Encoders:
[    2.812696] [drm]     CRT1: INTERNAL_UNIPHY3
[    2.812697] [drm]     CRT1: NUTMEG
[    2.924144] [drm] fb mappable at 0xC1488000
[    2.924147] [drm] vram apper at 0xC0000000
[    2.924149] [drm] size 9216000
[    2.924150] [drm] fb depth is 24
[    2.924151] [drm]    pitch is 7680
[    2.924428] fbcon: radeondrmfb (fb0) is primary device
[    2.994293] Console: switching to colour frame buffer device 240x75
[    2.999979] radeon 0000:00:01.0: fb0: radeondrmfb frame buffer device
[    2.999981] radeon 0000:00:01.0: registered panic notifier
[    3.008270] ACPI Error: [\_SB_.ALIB] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    3.008275] ACPI Error: Method parse/execution failed [\_SB_.PCI0.VGA_.ATC0] (Node ffff88042f04f028), AE_NOT_FOUND (20131218/psparse-536)
[    3.008282] ACPI Error: Method parse/execution failed [\_SB_.PCI0.VGA_.ATCS] (Node ffff88042f04f000), AE_NOT_FOUND (20131218/psparse-536)
[    3.509149] kfd: kernel_queue sync_with_hw timeout expired 500
[    3.509151] kfd: wptr: 8 rptr: 0
[    3.509243] kfd kfd: added device (1002:1313)
[    3.509248] [drm] Initialized radeon 2.37.0 20080528 for 0000:00:01.0 on minor 0
It is recommended to add udev rules:
# cat /etc/udev/rules.d/kfd.rules 
KERNEL=="kfd", MODE="0666"
(this might not be the best way to do it, but we're just here to test if things work at all ...)

AMD has provided a small shell script to test if things work:
# ./kfd_check_installation.sh 

Kaveri detected:............................Yes
Kaveri type supported:......................Yes
Radeon module is loaded:....................Yes
KFD module is loaded:.......................Yes
AMD IOMMU V2 module is loaded:..............Yes
KFD device exists:..........................Yes
KFD device has correct permissions:.........Yes
Valid GPU ID is detected:...................Yes

Can run HSA.................................YES
So that's a good start. Then you need some support libs ... which I've ebuildized in the most horrible ways
These ebuilds can be found here

Since there's at least one binary file with undeclared license and some other inconsistencies I cannot recommend installing these packages right now.
And of course I hope that AMD will release the sourcecode of these libraries ...

There's an example "vector_copy" program included, it mostly works, but appears to go into an infinite loop. Outout looks like this:
# ./vector_copy 
Initializing the hsa runtime succeeded.
Calling hsa_iterate_agents succeeded.
Checking if the GPU device is non-zero succeeded.
Querying the device name succeeded.
The device name is Spectre.
Querying the device maximum queue size succeeded.
The maximum queue size is 131072.
Creating the queue succeeded.
Creating the brig module from vector_copy.brig succeeded.
Creating the hsa program succeeded.
Adding the brig module to the program succeeded.
Finding the symbol offset for the kernel succeeded.
Finalizing the program succeeded.
Querying the kernel descriptor address succeeded.
Creating a HSA signal succeeded.
Registering argument memory for input parameter succeeded.
Registering argument memory for output parameter succeeded.
Finding a kernarg memory region succeeded.
Allocating kernel argument memory buffer succeeded.
Registering the argument buffer succeeded.
Dispatching the kernel succeeded.
^C
Big thanks to AMD for giving us geeks some new toys to work with, and I hope it becomes a reliable and efficient platform to do some epic numbercrunching :)