David works in daytime as a consultant for an IT company. After dark he works on his main interests, which include home and building automation.
13 stories

Representative Line: Off in the Distance

1 Comment

Drew W got called in to track down a bug. Specifically, their application needed to take a customer’s location, and measure the distance to the nearest National Weather Service radar station. It knew the latitude and longitude of each, and needed to find the distance between those points, and it was wrong. It could be off by hundreds or even thousands of miles, especially in more remote locations.

This was the code in question:

from math import sqrt
dist = sqrt((abs(latdiff) * abs(latdiff)) + (abs(londiff) * abs(londiff)))

Now, there’s an obvious problem here, and a number of nitpicks. I’m going to start with the nitpicks. First, when you multiply a number by itself, it’ll always be positive, so you don’t need the abs, making the line sqrt(latdiff*latdiff + londiff*londiff). Of course, Python also has an exponent operator, allowing you to write the easier-to-read version of, sqrt(latdiff**2 + londiff**2). But now that we bring it up, the math package in Python also includes a hypot function, which just implements the distance formula for you, meaning that whole thing could have been written thus:

from math import hypot
dist = hypot(latdiff, londiff)

Now, if your only criteria is, “which solution is more ‘pythonic’?”, then it’s clear that the latter solution is superior. Of course, you should still get your fingers whacked with a mechanical keyboard if you tried to check that solution in, because it still has one major problem: it’s completely and utterly wrong.

If you’re not sure why… think of it as a special kind of rounding error.

[Advertisement] Scale your release pipelines, creating secure, reliable, reusable deployments with one click. Download and learn more today!
Read the whole story
2770 days ago
Also known as the "what was the color of the bear" problem.
Geneva, Swizerland
Share this story

11/30/16 PHD comic: 'Academic Apps'

1 Comment and 5 Shares
Piled Higher & Deeper by Jorge Cham
Click on the title below to read the comic
title: "Academic Apps" - originally published 11/30/2016

For the latest news in PHD Comics, CLICK HERE!

Read the whole story
2779 days ago
Geneva, Swizerland
Share this story
1 public comment
2778 days ago
Rio de Janeiro, Brasil

How Do Software Developers in New York, San Francisco, London and Bangalore Differ?


When I tell someone Stack Overflow is based in New York City, they're often surprised: many people assume it's in San Francisco. (I've even seen job applications with "I'm in New York, but willing to relocate to San Francisco" in the cover letter.) San Francisco is a safe guess of where an American tech company might be located: it's in the heart of Silicon Valley, near the headquarters of tech giants such as Apple, Google, and Facebook. But New York has a rich startup ecosystem as well- and it's a very different world from San Francisco, with developers who use different languages and technologies.

Here on the Stack Overflow data team we don't have to hypothesize about where developers are and what they use: we can measure it! By analyzing our traffic, we have a bird's eye view of who visits Stack Overflow, and what technologies they're working on. Here we'll show some examples of what we can detect about each city based on one year of Stack Overflow traffic.

In this post we're going to focus on the four cities that visit Stack Overflow the most: San Francisco, Bangalore, London, and New York. (In this analysis, we counted all traffic within 50 miles of a city: this means San Francisco includes a larger part of the "Bay Area", such as Mountain View and Cupertino).

San Francisco vs New York

First we'll compare the two most popular American cities for software development: San Francisco and New York.

When developers are using a programming language or technology, they typically visit questions related to it. So based on how much traffic goes to questions tagged with Python, or Javascript, we can estimate what fraction of a city's software development takes place in that language.

For example, there were 187 million question views from San Francisco in the last year, and we can see that 12.8% of these visits were to questions with the Python tag, compared to 10.3% of New York's traffic.

Most of these common technologies look like they make up a fairly similar fraction of NY and SF traffic, but we're interested in stark differences. What tags (among the 200 most high-traffic tags) showed the largest difference between San Francisco and New York?

One clear difference: New York has a larger share of Microsoft developers. Many tags important in the Microsoft technology stack, such as C#, .NET, SQL Server, and VB.NET, had about twice as much traffic in New York as in San Francisco.

There are also patterns in the technologies that are more common in the San Francisco area, especially languages developed by Apple (Cocoa, Objective-C, OSX) and Google (Go, Android). We can also see several influential open source projects, especially ones associated with Apache (Hive, Hadoop, Spark).

Rather than looking only at the most dramatic changes, we could visualize the SF/NY ratio compared to the total visits:

This confirms that C# (in NY) and Android (in SF) stand out as the highest traffic tags that show different behavior, with tags such as Excel, VBA, Cocoa, and Go showing more even dramatic differences. Meanwhile, the Java tag has about the same level of traffic in each city, as do several "language agnostic" tags such as "string", "regex", and "performance".

New York, San Francisco, Bangalore, and London

Let's expand the story to include Bangalore, India, and London, England. Together these four cities make up 11.1% of all Stack Overflow traffic.

Each of these cities is the "capital" of particular tags, visiting them more than the other three cities do. Which tags does each city lead in?

This fills out more of our story:

  • London has the highest percentage of developers using the Microsoft stack: while New York had more Microsoft-related traffic than San Francisco, here we see London with a still greater proportion. Since both London and New York are financial hubs, this suggests we were right that Microsoft technologies tend to be associated with financial professionals.
  • New York leads in several data analysis tools, including pandas (a Python data science library) and R. This is probably due to a combination of finance, academic research, and data science at tech companies. It's not a huge lead, but as an R user in New York I'm still personally happy to see it!
  • Bangalore has the most Android development, with two to three times as much traffic to Android-related tags as the other three cities. Bangalore is sometimes called the "Silicon Valley of India" for its thriving software export industry, with Android development playing the largest role.
  • San Francisco leads in the same technologies as it did in the comparison with New York (except for Android). In particular (thanks to Mountain View), it's indisputably the "Go capital of the world." (This is true even if we look at the 50 highest-traffic cities rather than just the top 4).

This portrait of four major developer hubs is is just one of many ways Stack Overflow traffic can tell us about the global software engineers ecosystem. Whether you want to understand developers, hire them, engage them, or make your own developers more efficient, we have solutions to help you solve your problems. Check out Developer Insights to learn more.

Read the whole story
2780 days ago
Geneva, Swizerland
Share this story

Tech Companies, Fix These Technical Issues Before It’s Too Late


The results of the U.S. presidential election have put the tech industry in a risky position. President-Elect Trump has promised to deport millions of our friends and neighbors, track people based on their religious beliefs, and undermine users’ digital security and privacy. He’ll need Silicon Valley’s cooperation to do it—and Silicon Valley can fight back.

If Mr. Trump carries out these plans, they will likely be accompanied by unprecedented demands on tech companies to hand over private data on people who use their services. This includes the conversations, thoughts, experiences, locations, photos, and more that people have entrusted platforms and service providers with. Any of these might be turned against users under a hostile administration.

We present here a series of recommendations that go above and beyond the classic necessities of security (such as enabling two-factor authentication and encrypting data on disk). If a tech product might be co-opted to target a vulnerable population, now is the time to minimize the harm that can be done. To this end, we recommend technical service providers take the following steps to protect their users, as soon as possible:

1. Allow pseudonymous access.

Give your users the freedom to access your service pseudonymously. As we've previously written, real-name policies and their ilk are especially harmful to vulnerable populations, including pro-democracy activists and the LGBT community. For bonus points, don't restrict access to logged-in users.

2. Stop behavioral analysis.

Do not attempt to use your data to make decisions about user preferences and characteristics—like political preference or sexual orientation—that users did not explicitly specify themselves. If you do any sort of behavioral tracking, whether using your service or across others, let users opt out. This means letting users modify data that's been collected about them so far, and giving them the option to not have your service collect this information about them at all.

 Don’t collect it. If you have to collect it, don’t store it. f you have to store it, don’t store it long."

When you expose inferences to users, allow them both to remove or edit individual inferences and to opt out entirely. If your algorithms make a mistake or mislabel a person, the user should be able to correct you. Furthermore, ensure that the internal systems mirror and respect these preferences. When users opt out, delete their data and stop collecting it moving forward. Offering an opt out of targeting but not out of tracking is unacceptable.

3. Free up disk space and delete those logs.

Now is the time to clean up the logs. If you need them to check for abuse or for debugging, think carefully about which precise pieces of data you really need. And then delete them regularly—say, every week for the most sensitive data. IP addresses are especially risky to keep. Avoid logging them, or if you must log them for anti-abuse or statistics, do so in separate files that you can aggregate and delete frequently. Reject user-hostile measures like browser fingerprinting.

 "Would you rather be able to say “I don’t have any location data stored for my Muslim users”, or go to jail for refusing a subpoena?"

4. Encrypt data in transit.

Seriously, encrypt data in transit. Why are you not already encrypting data in transit? Does the ISP and the entire internet need to know about the information your users are reading, the things they're buying, the places they're going? It's 2016. Turn on HTTPS by default.

5. Enable end-to-end encryption by default.

If your service includes messages, enable end-to-end encryption by default. Are you offering a high-value service—like AI-powered recommendations or search—that doesn’t work on encrypted data? Well, the benefits of encrypted data have just spiked, as has popular demand for it. Now is the time to re-evaluate that tradeoff. If it must be off by default, offering an end-to-end encrypted mode is not enough. You must give users the option to turn on end-to-end encryption universally within the application, thus avoiding the dangerous risk of accidentally sending messages unencrypted.

Read the whole story
2787 days ago
Geneva, Swizerland
2788 days ago
Share this story

Star Trek Mad Science

5 Comments and 24 Shares
prokopetz: Random Headcanon:

That Federation vessels in Star Trek seem to experience bizarre malfunctions with such overwhelming frequency isn't just an artefact of the television serial format. Rather, it's because the Federation as a culture are a bunch of deranged hyper-neophiles, tooling around in ships packed full of beyond-cutting-edge tech they don't really understand. Endlessly frustrating if you have to fight them, because they can pull an effectively unlimited number of bullshit space-magic countermeasures out of their arses - but they're as likely as not to give themselves a lethal five-dimensional wedgie in the process. All those rampant holograms and warp core malfunctions and accidentally-traveling-back-in-time incidents? That doesn't actually happen to anyone else; it's literally just Federation vessels that go off the rails like that. And they do so on a fairly regular basis.

Getting a link to a whole thread on Tumblr seems to be an impossibility, but these are some amusing replies:

So to everyone else in the galaxy, all humans are basically Doc Brown.

Aliens who have seen the Back to the Future movies literally don't realise that Doc Brown is meant to be funny. They're just like "yes, that is exactly what all human scientists are like in my experience".

Vulcan Science Academy: Why do you need another warp core

Humans: We're going to plug two of them together and see if we go twice as fast

VSA: Last time we gave you a warp core you threw it into a sun to see if the sun would go twice as fast

Humans: Hahaha yeah

Humans: It did tho


Humans: It exploded twice as fast

Klingons: Okay we don't get it

Vulcan Science Academy: Get what

Klingons: You Vulcans are a bunch of stuffy prisses but you're also tougher, stronger, and smarter than Humans in every single way

Klingons: Why do you let them run your Federation

Vulcan Science Academy: Look

Vulcan Science Academy: This is a species where if you give them two warp cores they don't do experiments on one and save the other for if the first one blows up

Vulcan Science Academy: This is a species where if you give them two warp cores, they will ask for a third one, immediately plug all three into each other, punch a hole into an alternate universe where humans subscribe to an even more destructive ideological system, fight everyone in it because they're offended by that, steal their warp cores, plug those together, punch their way back here, then try to turn a nearby sun into a torus because that was what their initial scientific experiment was for and they didn't want to waste a trip.

Vulcan Science Academy: They did that last week. We have the write-up right here. it's getting published in about six hundred scientific journals across two hundred different disciplines because of how many established theories their ridiculous little expedition has just called into question. Also, they did turn that sun into a torus, and no one actually knows how.

Vulcan Science Academy: This is why we let them do whatever the hell they want.

Klingons: .... Can we be a part of your Federation

Let's talk about the USS Fucking Pegasus, testbed for the first Starfleet cloaking device. Here we have a handful of humans working in secret to develop a cloaking device in violation of a treaty with the Romulans. They're playing catchup trying to develop a technology other species have had for a century. And what do they do? Do they decide to duplicate a Romulan cloaking device precisely, just see if they can match what other species have? Nope. They decide, hey, while we're at it, while we're building our very first one of these things, just to find out if this is possible, let's see if we can make this thing phase us out of normal space so we can fly through planets while we're invisible.

"But why" said the one Vulcan in the room.

"Because that would fucking rule" said the Humans, high-fiving each other and slamming cans of 24th-century Red Bull.

Humans get mildly offended by the way they are presented in non-human media.

Like: "Guys, we totally wouldn't do that!" But this always fails to get much traction, because the authors can always say: "You totally did."

"That was ONE TIME."

There's that movie where humans invented vaccines by just testing them on people. Or the one about those two humans who invented powered flight by crashing a bunch of prototypes. Or the one about electricity.

And human historians go, "Oh, uh, this is historically accurate, but also kind of boring."

But when the Vulcans made first contact with Earth - "what the hell is that insane thing these aliens here have built, let's go look at it" - humans didn't look at them as an enemy or a resource or even an asset. No, the very first time humans met Vulcans, they tried to do the Vulcan hand thingy and they couldn't do it so they just offered a handshake, and then said "let's get drunk and party." THIS IS ACTUAL LITERAL CANON, REMEMBER.

Further in this vein:

I will never be over the fact that during first contact a human offered their hand to a vulcan and the vulcan was just like "wow humans are fucking wild" and took it

Note: Vulcan hand / finger touching is a sex thing.

My headcanon for startrek is that humans look, to vulcans, like a dog frathouse. Like signing on to a human ship is exactly that thrillingly loud and frustrating and fast and stupid and fun. The humans are going to dash off to a new sector to see if there are friends there and then they will jump up and down with delight and stuff their faces up against their new friends' genital array. The humans are going to bark for ten minutes at a rock. The humans want to chase things they can't possibly catch just because they like running around. The humans are madly passionate about their arbitrary group identities. The humans can be divided into new arbitrary group identities which they will then be passionate about. The humans want to stick their heads out of the window of their starship and go 'wheee!'. If you step on a human's paw they will act like you just killed them for about thirty seconds and then want more headpats. The humans can be immediately distracted from crucial duties by the appearance of a small animal. If you howl all the humans in earshot will howl louder just to show off. A human just humped your leg. 'Don't make it weird bro' the human says. Later the human will dig a weird bug out of the ground and eat it.

Previously, previously, previously.

Read the whole story
2787 days ago
Geneva, Swizerland
2788 days ago
Share this story
5 public comments
2785 days ago
Atlanta, GA
2786 days ago
I absolutely love this. It makes me proud to be human. We'll get to the Star Trek future someday, despite this year's little blip of an extinction bust. We're gonna be okay, folks.
2788 days ago
Boston, MA
2788 days ago
ha soo good.
Norfolk, Virginia
2788 days ago
Comedy Gold-Pressed Latinum.
New York, NY

Wear Your Uniform

1 Share

1995, Taiwan

Taiwan was going through a series of education reforms in the '90s. Our teachers started to incorporate new teaching methods inspired by the west in their curriculum. A 5.5-day working week was introduced, where students only had to work half a day on Saturday. It was a big deal.

On the other hand, although the shadows of World War II and Chinese Civil War were slowly dissipating, and people no longer lived in constant fear of invasion and aerial bombing, young men were still required to perform military service for two years once they reached their military age. Military officers would station at high schools to instill disciplines. It was socially acceptable for a teacher to physically punish students when they made a mistake.

When I enrolled in middle school, I had to have a crew cut while the girls had to cut their hair to shoulder-length. Our summer uniform was a black-and-white checkered shirt and short pants. Every year, our homeroom teacher would announce when to change into our winter uniform, which was a white long-sleeve dress shirt and dress pants.

2001, Japan

Muji, a large Japanese retailer, just saw its stock price plunge into an historical low amidst a pricing war with emerging competitors. Its overseas chain stores shut down, and profit declined. Tadamitsu Matsui, a former Human Resource and General Affairs manager at Muji, was promoted to head the company and right the ship.

One of Matsui’s proposals when he took over was to compile a 1,600-page manual on standard operating procedures, which he called “Mujigram.”

Matsui was a strong proponent of standardization. He famously noted that there are a hundred ways of doing one thing—if a Muji store manager is the one making all the decisions for their store, when they leave, it would mean the end of their store.

In just two years after Matsui took the helm, Muji was back on track.

By 2007, Muji was posting record sales of 162 billion yen and opened its first North American store in Soho, Manhattan.

2016, Brooklyn

Uniforms are the most visually effective way to show that we belong to a certain group. It helps with building our unity and promote our brand. But it is not enough just to wear them.

Before Matsui took over, individual Muji store managers had full control over what kind of store they wanted to run. As a brand, however, Muji was losing its identity. Their long-time customers could not recognize whose product they were buying from—even if the store clerks were all wearing their uniforms.

In my youth, a uniform was a piece of clothing that I wore to school every day. I was wearing what my school and society deemed fitted for a student. But more than a piece of clothing, it was the product of the accumulation of history and culture that took place before my time.

Organizations that understand culture not only have uniforms, they have principles. Principles that permeate through our outward appearance, behaviors, and processes. I am fortunate enough to work for such a company.

We invest in our people disproportionately when compared to the rest of the industry. We train each new hire through a rigorous apprenticeship program, which we pay for, and have continuously improved over the past 10 years.

We ensure every one of our employees share the same set of professional values through our apprenticeship program and know what we stand for.

What do others see when they look at your team?

Read the whole story
2793 days ago
Geneva, Swizerland
Share this story
Next Page of Stories