Archive for the ‘Internet’ Category

Verizon’s Morse Code Post, Translated

Thursday, February 26th, 2015

Today the FCC ruled in favor of Net Neutrality. Opponents such as Verizon, thought this was an antiquated decision. So in what is probably the most childish response by a corporation I’ve seen to date, Verizon responded with a blog post stating their disappointment at the FCC’s decision. In Morse Code. They also provided a link to a PDF. With typewriter-style-smudged-ink text. Dated Feb. 26, 1934.

All right Verizon, challenge accepted.

I copy-pasted their blog post and ran it through a short python script I made. This is what comes out (formatting mine).

“Today’s decision by the FCC to encumber broadband internet services with badly antiquated regulations is a radical step that presages a time of uncertainty for consumers, innovators and investors. Over the past two decades a bipartisan, light-touch policy approach unleashed unprecedented investment and enabled the broadband internet age consumers now enjoy. The FCC today chose to change the way the commercial internet has operated since its creation. Changing a platform that has been so successful should be done, if at all, only after careful policy analysis, full transparency, and by the legislature, which is constitutionally charged with determining policy. As a result, it is likely that history will judge today’s actions as misguided. The FCC’s move is especially regrettable because it is wholly unnecessary. The FCC had targeted tools available to preserve an open internet, but instead chose to use this order as an excuse to adopt 300-plus pages of broad and open-ended regulatory arcana that will have unintended negative consequences for consumers and various parts of the internet ecosystem for years to come. What has been and will remain constant before, during and after the existence of any regulations is Verizon’s commitment to an open internet that provides consumers with competitive broadband choices and internet access when, where, and how they want.”

“Verizon’s commitment to an open internet that provides consumers with competitive broadband choices” Really, Verizon? Really?

The Mexican GovBots Did NOT Take Down #YaMeCanse, But We Can Keep #YaMeCanse# Trending

Monday, December 8th, 2014

Perhaps it was the excitement of hearing about a new phenomenon in censorship that prompted me to write a little too hastily about how the government of Mexico might have used Twitter bots to spam and trash the #YaMeCanse hashtag out of the trending topics list. As reported by Lo Que Sigue, Sopitas.com, Aristegui Noticias, and myself, #YaMeCanse, the hashtag used as the rallying cry for Mexico’s 43 missing students, was suddenly dropped from the trending topic list by an army of bots, presumably coordinated by the federal government. No proof is provided by any of us that the government was behind this, but a series of videos and screenshots originally provided by Lo Que Sigue lead us to believe that a swarm of bots is at least responsible.

It was NOT the bots

December 3 at 10:36 AM was the last time @TrendieMX reported #YaMeCanse to be trending. By 9:36 PM of the next day, #YaMeCanse2 was already trending. Let’s take a look at what the Topsy trends for #YaMeCanse looks like for the month of November and the first days of December.

Usage of #YaMeCanse

 

To make sense of what happened, we need to understand what Twitter is doing to calculate a trending topic. We don’t have access to specific information about how the trending algorithm functions, but we do know how trending algorithms in work general and we have some clues about what Twitter has done in the past to tweak its algorithms. The relevant issue here can be described as “the Justin Bieber problem”. Many of you might remember how some years ago Justin Bieber was constantly trending due to the millions of Beliebers continuously tweeting about him. Twitter wants to tell us what’s trending right now, and not one hour ago or one month ago. As Twitter is quoted saying in this Mashable article:

“The new algorithm identifies topics that are immediately popular, rather than topics that have been popular for a while or on a daily basis, to help people discover the ‘most breaking’ breaking news from across the world. (We had previously built in this ’emergent’ algorithm for all local trends, described below.) We think that trending topics which capture the hottest emerging trends and topics of discussion on Twitter are the most interesting.”

Instead of merely looking at volume of Bieber tweets (of which there are many), Twitter looks at speed and “burstiness” of the tweets. However, there’s more to it. If Twitter only measured “burstiness”, you might see “Good Morning” trending every morning of every day. For this, Twitter establishes a baseline of expected frequencies based on history. Twitter “knows” there is usually a spike of “Good Morning” tweets every morning and corrects for this. As this video on trend detection in twitter social data explains, a ratio is calculated for each term based on the past frequency of the term and the present frequency.

What most likely happened is that after a couple of weeks of trending, the baseline for #YaMeCanse rose from zero (it didn’t exist before 11/7) to the frequency of people tweeting at the end of November. Twitter treated the volume and speed of the hashtag as something it would expect and dropped it off the trending list.

Baseline shift on #YaMeCanse

Spam bots should have no impact on the algorithm. The spam team at Twitter identifies the bots and they are not counted towards the algorithm. Additionally, it’s worth mentioning that Twitter has a team of low-paid human workers manually sorting through hashtags to eliminate advertiser spam. Even so, there is no evidence of an increase in bots during the time the hashtag was dropped from the list. The team at Lo Que Sea provided this video as proof of the presence of bots (not that we need proof of that in general)

Screenshot of Lo Que Sea video

Why are individual and unconnected tweets labeled as bots? If I tweet and only one person RT’s me, by their standards, I’m a bot. You can run the simulation from the video yourself on flocker.outliers.es. Use #YaMeCanse2 and wait for the same pattern of connected and disconnected tweets to occur. Then zoom in on the disconnected tweets and look up a couple of usernames. You’ll find a lot of those disconnected nodes are real people. You’ll also run into bots, but having no one retweet your tweet does not make you a bot.

This has happened before.

This would not be the first time that people have cried censorship upon the disappearance of a hashtag from the TT list. The Mashable article quoted above was a response to Beliebers accusing Twitter of censorship. Similarly, occupiers accused Twitter of censorship when #OccupyWallStreet was taken off the list. In both occasions Twitter had to step in and say this was just a result of how the algorithm works. In some cases we should be glad the algorithm works like this, otherwise we’d see #JustinBieber constantly trending. But how about when it’s something important like #YaMeCanse?

At this point I should say that if it were possible for the Mexican government to use such a tactic to censor people on social media, they probably would. We’ve already seen how Peña Nieto’s campaign used bots to promote the candidate on Twitter. And earlier this year, an initiative put forth by Peña Nieto on Radio and Telecom caused a lot of controversy when people claimed the law would allow the government to censor online content and to interrupt cell reception during protests. There’s also the case of 1DMX.org which was censored by GoDaddy under pressure from the US Consulate in Mexico.

We’ve Discovered How To Get Around the Algorithm

I believe the immediate response by the Mexican Twitterverse in the creation of #YaMeCanse2 has revealed an exploitable feature in the algorithm. It took less than two days for people to adapt to the new hashtag. Topsy Trends shows that #YaMeCanse2 doesn’t have significantly more traffic than #YaMeCanse had before being taken down, but the reason why #YaMeCanse2 was able to trend so quickly is because its baseline at the time was zero. This means whenever #YaMeCanse2’s baseline shifts up enough for it to de-trend, we can just start again with #YaMeCanse3. We can keep going with this as long as we keep the speed at which people tweet constant or as long as Twitter doesn’t catch on and modifies the algorithm to account for us just adding a number at the end of the phrase (In which case we can just add the word “tres”). This is also why we keep seeing so many different Bieber hashtags, they’re all different phrases that didn’t exist before.

The lesson here for the Mexican folk is that if we want to continue to have a #YaMeCanse hashtag trending, we need to coordinate to increment the number at the end of the tag each time it expires. When #YaMeCanse2 falls off the list, we simply switch over to #YaMeCanse3.

Reddit is NOT a failed state….

Thursday, October 9th, 2014

It has it’s problems for sure, but I wouldn’t be so quick to dismiss it as having failed.

I’m referring to a The Verge article posted about a month ago following the celebrity nude photo leaks. The main argument for FAIL is the fact that instead of chastising the users who help spread the leaked photos, Reddit protected them under the shield of free speech. I’m not here to argue whether Reddit acted appropriately or not in protecting the individuals (personally, they could’ve been kicked out, banned, arrested, and I would’ve been content with that). But I do not think this transgression in privacy, abuse of free speech, and overall disgusting behavior by a small group of a larger community a failed state makes.

Is this indicative of pervasive malicious behavior across Reddit? Absolutely. We didn’t need r/TheFappening to figure that out. Just talk to woman redditors about their experiences as participants.

But at least we’re talking about these issues. It’s not so much the fact that we are, it’s the fact that we have the ability to do so. Through their karma system, Reddit has built a system that promotes good behavior and–sometimes–reproves the bad. It’s a primitive system for sure, especially since it’s not immune to the hivemind behavior (for example, apparently the r/nyc hivemind thinks people have ZERO responsability to give up your seat in the subway for a pregnant woman (maybe they’re right and I’m wrong)). This system, I think, allows the hive to go through iterations of what they believe to be correct. In effect, every now and then it corrects itself. Take the terrible “detective work” conducted during the immediate aftermath of the Boston Marathon bombing. After the hive realized it was wrong (so wrong), whenever there was a post asking for some sort of crowdsourced detective work, it was often met with someone who commented on the terrible results that came out of the last time they tried to play detectives.  As a result, Reddit for the most part now knows: We should avoid digital vigilantism.

In the coming years we will increasingly see Nobel Prize winner Elinor Ostrom’s principles on governing the commons applied to digital spaces. Although primitively (and perhaps unintentionally) Reddit has created a space where communities are able to define their own boundaries, (sort of) align “governance” rules with their preferences, (kind of) ensure that those who participate in the community can have a say on the rules, and are (barely) able to sanction those who misbehave. It has a long way to go for sure. What happened with r/TheFappening is a case where a group of very misguided individuals were able to gather in one place and as a community behave inappropriately. In that case what Reddit might be lacking is some greater oversight over communities and their leaders. An oversight that’s not dictatorial, but rather an oversight that is also provided by a community (a council of communities?).

Another problem with Reddit (or any digital space, actually) is that whenever someone goes through the trouble of committing a crime–say stealing nude celeb photos–the “morality cost” of engaging in the immoral behavior is significantly decreased by the internet’s ability to massively distribute information at a significantly low cost. For the most part, the consequences for engaging in such immoral behavior do not exist. Especially when it costs nothing to click on a link. This is maybe one of the internet’s biggest weaknesses: it’s ability to facilitate engagement in immoral behavior.

We need to design digital spaces that somehow take this into account. Spaces where the community can more meaningfully participate and deal with the bad apples more effectively. Is Reddit, and the rest of the internet, full of misguided individuals who do some fucked up shit? Yes, but this doesn’t mean we need to take it to the back of the barn and shoot it. It means we need to think about how we create these digital spaces in the future. Or do away with it if you want, but then let’s take the good lessons and the bad, and let’s make something better.

 

How to Build a Website From Scratch

Thursday, January 23rd, 2014

When I signed up to build the Open Data 500 website, I wanted to go through the entire process of making a website from scratch. Full stack. Just to sort of see what it was like.
After spending 5 entire 10-hour days trying to trouble shoot a feature on the site, I decided to write a post on the skills needed to build an entire website from scratch.

To build an entire website from scratch you need to know the following:

  • HTML5
  • CSS
  • JavaScript
  • jQuery
  • D3
  • ParsleyJS
  • Modernizr
  • Tornado
  • Python
  • MongoDB
  • Mongoengine
  • CSV
  • JSON
  • geoJSON
  • Regular Expressions
  • Seamless
  • Heroku
  • Command Line
  • Git / Github
  • Google Analytics
  • MailChimp
  • DNS Records (A, CNAME, MX, etc)
  • Oh yeah, go directly to hell, GoDaddy
  • Polar Vortex Survival Skills
  • Basic Pharmacology
  • UX
  • UI
  • FU
  • F712U
  • Scheme (might as well)
  • Creative Commons Licensing
  • PHP (throw in a couple more languages, just in case)
  • Java
  • Ruby
  • C#
  • C♭
  • Perl
  • .NET
  • Obviously not WordPress
  • Ballmer Peaks
  • Double-team keyboarding
  • Windows
  • Mac
  • Linux
  • Atari
  • SNES (you’re welcome)
  • Brainfuck
  • SSL
  • HTTP
  • API’s
  • SOAP
  • LDAP
  • TCP/IP
  • WOFF
  • DOM
  • Cookies
  • XSRF
  • RSS
  • XML

I think that’s about it. I’f you’re just beginning with web development. Good luck. You’re almost there.

26

(Seriously, though. Keep it up, the road is long and arduous, but it’s totally worth it)

 

 

Pushing Policy Through Mexican Telenovelas

Wednesday, August 14th, 2013

In recent Mexican news, there has been some talk about an energy reform. One of the biggest and most contentious issues being discussed is whether the state-owned oil company, PEMEX, should be allowed to receive foreign investment. Whether it should or it shouldn’t is not the topic of this post.

A couple of months ago, a YouTube user uploaded this clip from the telenovela Corazón Indomable.

Translating, and attempting to keep the tone of the conversation, the dialog goes something like this:

Woman 1: “Is there an inconvenience ma’am?”

Woman 2: “I didn’t think that a foreigner could have property in this island.”

Woman 1: “Yes, we are happy that is so. We would be sad if we didn’t have investment from abroad.”

Woman 2: “Why?”

Woman 1: “Because places prosper with everyone’s talent, national and foreign.”

Woman 2: “Hmm, but foreigners take the money with them, no?

Woman 1: “Money comes and goes, what you’re talking about with the Emir, he practically leaves it here because he spends it here.”

[Some talk about the Emir.]

Woman 2: “Well, every day you learn something new.”

Woman 1: “And what did you learn today?”

Woman 2: “Well, that foreign investment is really necessary and convenient.”

Woman 1: “Don’t doubt it. Without disparaging the national ones, of course.”

Woman 2: “Well, thanks for everything and good day.”

Woman 1: “Good day, excuse me.”

This telenovela is produced by Mexico’s largest media company (and largest in the Spanish-speaking world), Televisa. Televisa generated a lot of controversy during last year’s presidential election because of an alleged secret collaboration between the PRI and Televisa. The media company has been criticized for it’s low quality content and often ridiculous telenovelas (if you don’t speak Spanish, I’m sorry, you’re missing out), but I’m not sure I have seen this blatant use of telenovelas to convince the public of a political agenda before. It even gets didactic in the end when the woman asks “And what did you learn today?”

In a country where only two companies (Televisa and TV Azteca) own practically all TV content in Spanish, it is becoming increasingly important to make sure more people have access to the internet, where they would have access to a broader variety of media content, and as the Soraya meme shows, would even have the ability to ridicule such horrible TV .