Bots Are People Too

November 27th, 2015

I came to love bots because of @TwoHeadlines, @CongressEdits and NYTimes Haiku. Not only did they bring me great entertainment, but as is the case with @CongressEdits, I thought they were providing a great service. However, not all bots are cool. Lurking among us are bots sponsored by governments. Bots whose mission is to give a false impression of support or to drown out opposing views. These types of bots have been used in recent years by the governments of Mexico, Venezuela, Russia and Syria. However the method by which these governments use bots is not as clear as are political bots like @CongressEdits. In this post, I’d like to use the government of Mexico’s use of political bots as an example of how the traditional definition of political bots might not be enough cover important use cases.

I’ll start by using Leivetz’s definition (via Savage) of bots. Bots are “software applications that perform repetitive tasks automatically or on a schedule over the internet, tasks that would be too mundane or time-consuming for an actual person.” I like this definition because it leads me to ask the question: What if you had access to enough resources and low-skilled labor such that you could perform the tasks of bots with real people? This is how our story of the Peña bots begins.

Shortly after the start of the 2012 Mexican presidential election, then candidate Enrique Peña Nieto rose to Twitter dominance. It seemed the candidate was enjoying an unusual amount of support from social media users. To some savvy netizens, this seemed suspicious. Sure enough, Twitterverse eventually discovered that the EPN campaign was making use of large armies of volunteers to show support for the candidate and to drown out any opposition that could appear online. The name Peñabot appeared shortly after this discovery in November 2011. It is unclear whether the word “bot” in “Peñabot” was originally used to fit Leivetz’ definition of a “software that performs a repetitive task” or if it was originally used to refer to actual volunteers who under direction from campaign managers would mindlessly perform repetitive tasks such as tweeting the same text and retweeting other supporters. In case of the latter, the word Peñabot is similar to Limbots and Obamabots and is often used in parallel with Pejezombies, or the followers of competing candidate Lopez Obrador.


Although people seem to be pretty confident that the campaign (and now actual government) used software bots, it has been confirmed that EPN did in fact use armies of volunteers to take over the conversation on Twitter. One former EPN social media manager confessed that at one point the campaign was coordinating 20,000 volunteers to show support for the candidate and to target and drown out any opposing hashtag that could surface during the campaign trail.

The ability to coordinate such a large army of volunteer can be effective in bumping hashtags off the trending topic list since Twitter’s algorithms measure “burstiness” of tweets. One example where volunteers overran a hashtag was with #MarchaAntiEPN (March Against EPN) in the state of Tabasco where volunteers bumped the hashtag off the list with #TodoTabascoConEPN (All of Tabasco With EPN). People discovered the presence of Peñabots because the majority of #TodoTabascoConEPN tweets came from a single location in Mexico City (apparently tweets were geotagged?).

There is no reason why this can’t be done using actual software bots (RT’ing prog-government tweets, tweeting from a list of pre-written messages, crowding hashtags and making your own hashtag trend). This is why I wonder if a large army of volunteers acting in coordination from the government can be classified as bots? Should this strategy be included in conversations about political bots?

Human or Bot?

A Brief History of Narcocorridos

October 20th, 2015

Unless you live in northern Mexico or you’re a Mexican-American living in any of the border states, chances are you’ve never heard of American singer-songwriter Gerardo Ortiz. And yet, if you were to measure popularity by an artist’s YouTube views and subscribers, then Gerardo Ortiz would be twice as popular as Madonna. His 2013 hit “Dámaso” from the Latin-Grammy nominated album “El Primer Ministro” has over 140 million views on YouTube. Gerardo Ortiz sings narcocorridos, Mexican folk ballads about famous drug lords, their exploits, lifestyle, and the drug trade in general.

Ortiz is just one of hundreds of narcocorrido artists that have become incredibly popular in the past decade. Another such artist is Alfredo Rios, better known as El Komander who is part of a sub-genre of narcocorridos called ‘movimiento alterado’, or ‘altered movement’. Unlike the narcocorridos, these songs go beyond chronicling the lifestyle and adventures of drug dealers and instead go into very explicit detail on beheadings, torture, and kidnappings that are so commonplace among the narcos.

Con cuerno de chivo y bazuka en la nuca
volando cabezas al que se atraviesa

With an AK-47 and a bazooka around my neck
blowing off the heads of whoever gets in our way

Where did this music come from and how did it become so unbelievably popular among millions of people? Narcocorridos have been around for a long time and have gone through several important transformations, and as their popularity increased, it came to draw the attention of authorities. The glorification of the violence and the narco-lifestyle has been major pain in the side of the Mexican authorities who have repeatedly sought to censor such music. This attempt at censorship has drawn a lot of comparisons between narcocorridos and gangster rap from the 1980’s. El Komander has even been called “the Jay Z of Mexican drug balladeers.

Corridos have a long history in Mexico, starting from the Mexican War of Independence in 1810 and throughout the Mexican Revolution (1910-1921). The ballads usually told stories about oppression by the government and usually carried hidden political meanings as was common during the revolution. They were also about the daily lives of peasants and sometimes about a lost or disgraced love. In fact, you’ve probably heard one of these corridos before.

Ballads about smuggling appeared in the late 19th century, when the government of president Porfirio Diaz imposed tariffs on fine textiles upwards of 300% making the smuggling of textiles from the US into Mexico a very lucrative business. One of the most famous textile smugglers was Mariano Resendez, the subject of one of the first ballads about contraband. Resendez, like the subjects of many corridos, was considered a hero among the communities in Northern Mexico because of his resistance to government forces. In fact, drug lore is full of such characters. Most famous of which is Jesus Malverde, angel of the poor, patron saint of the narcos, and subject of many narcocorridos.

Although his existence is not verified, Malverde was most likely not a drug smuggler since marijuana and opiates were not illegal during his time (b. 1870 – d. 1909). It wasn’t until 1920 and 1925 that Mexico prohibited the sale and use of marijuana and opiates, respectively. What’s significant about Malverde is his Robin Hood-esque fame. His story is said to be modeled after Heraclio Bernal (1855-1888) who, quite literally, stole from the rich and gave to the poor. This important trope will remain a significant part of narcocorridos for many decades.


Jesus Malverde soap

Photo by: David Agren

It wasn’t until the 1920’s with the enactment of the 18th Amendment and the subsequent Volstead Act that smuggling really took off. Smugglers known as Tequileros would push rafts full of–you guessed it–Tequila across the Rio Grande and would drive trucks through the desert to smuggle it into the US. In addition to smuggling, corridos at the time also dealt with social issues. As “El Corrido de los Bootleggers” demonstrates, the dire conditions in some parts of Mexico sometimes left people with no other choice but to resort to smuggling.

Me puse a pensar señores
Que trabajo ya no había
Tenía que buscar mi vida
Si el señor me concedía

I started thinking, gentlemen
There is no more work
I have to find my life
If the Lord would grant me

When prohibition ended in 1933, the Tequileros suddenly found themselves out of a job and turned to other controlled substances. Drug smuggling took off shortly afterwards and with that came the first drug ballads. One of the first narcocorrido is said to have been “El Contrabandista” written by Juan Gaytán of the “Gaytán y Cantú” band. It perfectly illustrates the shift in the smuggling business.

Comencé a vender champán, tequila y vino habanero,
Pero este yo no sabía lo que sufre un prisionero.
Muy pronto compré automóvil, propiedad con residencia,
Sin saber que en poco tiempo iba ir a la penitencia.
Por vender la cocaína, la morfina y la mariguana,
Me llevaron de prisionera a las dos de la mañana.

I began selling champagne, tequila, and Havana wine,
But I did not know what a prisoner suffers.
Soon I bought an automobile, property with a house,
Without knowing that in a short time I would be going to jail.
For selling cocaine, morphine, and marijuana,
They took me prisoner at two in the morning.

After the initial appearance of drug ballads, there seems to be a gap between the 1950’s and 1960’s, when nothing major happened in the narcocorrido space. This could have been due to the economic prosperity Mexico enjoyed during these decades. The resurgence of narcocorridos seems to coincide with the social and economic trouble that enveloped the country at the end of the 1960’s. Another major factor for their resurgence could have been the increase in marijuana consumption in the US during the 1960’s or the increase in heroin traffick due to the collapse of its market in Europe and the Middle East. Whatever it was, the definite turning point for narcocorridos came in 1972 when a song about a now very famous Texan hit the airwaves. The song, by norteño band from Northern California, would mean the rebirth of a genre that would grow to reach millions of listeners on both sides of the border.

Ángel González wrote “Contrabando y Traición” in the mid 60’s and Los Tigres del Norte, a band formed earlier in San Jose, CA recorded it in 1972. The story begins with Camelia “La Texana” travelling north from San Isidro to LA with her partner Emilio Varela, their car tires filled with marijuana. Upon arriving to Hollywood and finalizing the drug deal, Emilio Varela tells Camelia he is leaving her for another woman in San Francisco. Camelia does not take this news too lightly and shoots Emilio, taking off with the money and never being heard of again. Up until then, most of the corridos had been about real events, but Ángel had completely made up the story of Camelia (even though he did happen to know a Camelia who isn’t from Texas and an Emilio Varela, his cousin, who had never met Camelia). In writing a fictional song, Ángel set the genre on a new path. Soon after the song’s widespread success, hundreds of writers tried to replicate the formula and it took the popularity of narcocorridos to new heights.  

In an interview by author Elijah Wald, González tells how after 30 years he has not been able to come to terms with the song’s enormous popularity and the countless of corridistas it inspired. “That song, I wrote it without thinking, I had no idea what would happen afterward. After my corrido, along came that whole pile of songs about drug traffickers, but I wrote it without any idea of that. It was a problem I brought to light, but not something I knew much about […] I never, never ever thought that the song would make it big.” While it seems like González had a strong moral stance against drug smuggling, it is not clear that the rest of the corridistas and their fans shared the same ideals.

Researcher, Juan Carlos Ramirez-Pimienta says drug trafficking was seen almost as a patriotic duty since it was bringing large quantities of money into Mexico. The economic benefits of the drug trade are of course exaggerated, he says, but it’s not difficult to see how in the minds of a lot of people, drug trafficking was not all that bad, especially during Mexico’s troubled economy. What’s more, the cartels would often give out “narcolimosnas”, or narco-charity to win over the public’s opinion. It’s almost too obvious that the major drug lords would engage in the same type of generosity as legends like Jesús Malverde. If the lower classes in Mexico had the social, cultural, financial, and even military backing from the government, it would have been harder for the cartels to win over large segments of the Mexican population (decades later, this sort of justification would no longer be necessary with the “Movimiento Alterado”).

With or without moral justification, the genre exploded and corridos underwent several transformations. The songs shifted from portraying the protagonist as someone who just smuggled drugs into someone who reaped and enjoyed the riches it brought. The smuggler was now someone who consumed drugs and not just smuggled them, someone who enjoyed the lifestyle, someone who through their own brawn rather than their brain rose up from poverty to achieve great success in the form of riches and power. Researcher Helena Simonnet says:

“The changing social reality of Mexico’s Northwest also affected popular music, notably the corrido production. Although the image of the brave man that was generated by the Mexican Revolution in the early twentieth century still holds for the protagonists of the contemporary corridos, the meaning of bravery has changed. While the heroes of the folk corridos raised their arms for social justice and equality, the tough guys of the narcocorridos carry their weapons for personal enrichment and empowerment.”

No one embodies the quintessential narcocorrido protagonist of the time better than Rafael Caro Quintero. Born in a poor rural town in Sinaloa, Caro Quintero went on to be a billionaire as the founder of the Guadalajara Cartel. His story alone could fill volumes as it involves the CIA, the DEA, the Contras, and billions of dollars stashed in Swiss bank accounts. In the eyes of the people, Caro Quintero was a hero. For one, he was born into a very poor family where he was one of 12 siblings. In his teenage years he began growing and selling marijuana, amassing a small fortune that would later become billions. He also defied the US government in a way that resonated among people living along the border, people who had suffered at the hands of US authorities. And of course, he was known to be very generous having funded large infrastructure projects such as a highway in his hometown of Badiguarato. Hundreds of songs were written about Caro Quintero.

At the same time as corridos were exalting narco-heroes for their generosity, they were also celebrating the lavish narco lifestyle. Mexican authorities, out of fear that these songs would act as recruitment strategies for the cartels, began cracking down and in some places banned these songs from the airwaves. This never proved fruitful since much of the audience resided in the US where freedom of speech laws protected the corridos from any sort of censorship. What’s more, US radio stations have no regard from borders, so people who lived along the border on the Mexican side could still tune in to American radio stations. Censorship even proved advantageous to bands such as Los Tigres del Norte who capitalized on prohibition by marketing their albums as “forbidden”. Despite relatively low airplay in Mexico, their best-selling album “Corridos Prohibidos” (Forbidden Corridos) was hugely successful even though there was nothing “forbidden” about their songs, especially not in the US. Their strategy proved so successful that many other bands followed suit like Los Tucanes de Tijuana’s “Tucanes de Plata: Catorce Tucanes Censurados”.

As the popularity and demand for narcocorridos raged on, in the early nineties, a new phenomenon began to take hold of Los Angeles nightclubs. Many capos, having heard many commercial narcocorridos about the most famous drug lords such as Caro Quintero, decided they wanted to be protagonists of their own corridos, so they started commissioning their own.

Among the most influential commissioned-corridistas was Rosalino “Chalino” Sánchez. As the story goes, Chalino was born in 1960 in rural Sinaloa. When he was five years old, a local tough guy raped his sister. Ten years later, Chalino ran into him at a party and without saying a word walked up to him and shot him dead. Soon afterward he fled Sinaloa and moved to LA to live with his aunt. In LA he would work several jobs including low-level smuggling which would connect him to the narco-world. In 1984, his brother, Armando, was shot and killed in Tijuana and Chalino would write his first narcocorrido to preserve his memory. Soon afterward he found himself serving jail time where he would write songs about his fellow inmates in exchange for money or favors. Word spread of his writing talents and after his release he found himself in high demand among low-level narcos who would pay him cash to produce cassettes with their ballads performed by a band. While still a relatively local phenomenon in southern California, his big publicity break came in early 1992 when, while singing at a club in Coachella, someone came up to the stage and shot Chalino on his side. Chalino pulled out his gun and shot back. By the end of the shootout, the would-be assassin would end up dead, shot in the mouth with his own pistol. News coverage of the shooting made him a sensation on both sides of the border. Months later, Chalino’s luck ran out when after a concert in Culiacán he was killed after being picked up by a group of armed men. He became the Mexican version of Tupac Shakur. Chalino’s death elevated him to legendary status. Hundreds of songs were written about him, his song “Nieves de Enero” became a radio hit, and soon the narcocorrido genre was flooded with imitators.

The narcocorrido trend continued throughout the nineties. While Mexico was enjoying a period of relative peace, so were the corridos. All that seemed to change when Felipe Calderón took office in 2006. Soon after Operation Michoacán in December of the same year, the country turned violent and so did the corridos, and with that came the birth of the Movimiento Alterado. Although now used as a generic term, Movimiento Alterado (“altered” movement, as in the altered state of your brain, supposedly) was a product of Adolfo and Omar Valenzuela, owners of Twiins Music Group who produced a lot of corridos alterados through their subsidiary La Disco Music. Unlike narcocorridos before them, this new breed of corridos was hyper violent, with lyrics about blowing people’s heads off and killing those who deserved it.

Even though there is a clear difference between older corridos and new corridos “alterados”, artists from this new movement seem to disagree with critics who say their songs promote violence. Singer-songwriter El Komander has said during interviews his songs are about partying and about having a good time, not about promoting violence. But the truth is that the festive corridos El Komander describes have been around since the nineties and it is very difficult to find any songs from that decade that match the level of violence contained in the new corridos.

While it was Calderon’s bloody war that caused the birth of this new genre, it was the increasing adoption of the internet in Mexico that allowed for its spread. The low production costs and essentially free distribution afforded by services such as YouTube allowed hundreds of young people to take risks by exploring new styles and helped them gain new audiences. In the late 1980’s Chalino had to sell cassettes out of the trunk of his car. Nowadays singer-songwriters only need a camera and an internet connection. The internet even serves as a source for inspiration. In the documentary film “Narco Cultura”, BuKnas frontman Edgar Quintero, who lives in LA but seldom travels to Mexico, watches YouTube videos from Mexico to draw inspiration for his own songs.

It is worth noting that at this point in time, not only did artists merely sing about violence, but the violence also came to them in a very real way. Chalino Sanchez popularized the commissioned narcocorrido trend. Unfortunately, he might have also been the first in a long list of ill-fated corridistas. Singing about the narcotrafficking world turned out to be an extremely dangerous profession.

All of these people were killed between 2006 and 2010. During the same period, the Committee To Protect Journalists lists 38 killed journalists. In Mexico, being a narcocorrido singer, is almost as dangerous as being a journalist.

During the past 15 years, narcocorridos have become harder to categorize. They are more complex, dealing with a variety of subjects. Some songs give people an insight into the secret and underground world of narcotrafficking. Others very graphically describe the exploits of the famous capos. Then there are others which are purely about the narco-lifestyle. Narcocorridos are many things: stories, eulogies, praise, and even news sources.

In a country where traditional media is censored via intimidation, through the government, or even self-censorship, millions of people are forced to resort to alternative news sources such as El Blog del Narco. This blog and it’s many re-incarnations and copycats have been extremely popular sources for people who want to get the inside scoop into world of the narcos. Narcocorridos offer people another peek into the secret world of narcotrafficking. Narcocorridos don’t need to act as a source of breaking news to be relevant. They are relevant because they are an important vehicle for distributing information about major events. One recent example is Joaquín “El Chapo” Guzman’s second escape from prison. Within hours, corridistas had written, composed, recorded, and uploaded their songs on the internet about El Chapo’s escape.

El Chapo’s escape was widely covered in the news, but a much less covered news story at the time was the death of Atanacio Torres Acosta, son of Manuel Torres Felix “El M1”, a high-ranking member of the Sinaloa Cartel in 2008. His death, which was the result of a rivalry between M1 and the Beltrán Leyva Cartel, left a trail of torture and murder for many weeks and solidified the rivalry between the cartels that would leave thousands dead for years to come. In April of 2008, gang members of the latter killed Torres Felix’s son, Atanasio and injured his 4-year-old sister (apparently she lost half of her right arm). The killing sent Torres Felix into a killing spree, which earned him the nickname “El Ondeado” (The Crazy One). About a month after the killing, two dead bodies appeared in the same location, one of which was found decapitated inside a plastic barrel. A sign next to the bodies read: “These are the homosexual gunmen of Arturo Beltrán Leyva and the guy in the barrel is non other than the childkiller ‘El Caimán’. So you learn to respect. My respects for the children.” At the time very few sources covered the newsHowever the story is now part of narco-lore partly because of the song “La Venganza del M1”:

Cobro justo liquido las cuentas
de la empresa del m y la z
conocido como el m1

es mi vicio la sangre enemiga
la venganza se me hizo un placer
habra torturas, habra un infierno

el 4 fue valiente mi hijo
fue abatido por unos cobardes
tenian miedo de llegar conmigo

con un pie presionaba su pecho
con una mano le agarro el pelo
en la otra mano trae un cuchillo
lo decapito les corto el cuello
y dejo junto a el un mensaje
que para los niños su respeto

Charged justly, settled accounts
Of the M and Z enterprise
He is known as the M1

My vice is enemy blood
Vengeance is my pleasure
There will be tortures, there will be hell

4 was brave, my son
Was killed by some cowards
They were afraid to meet me

With one foot I pressed against his chest
With one hand I grabbed his hair
In the other hand, a knife
I decapitate him, I cut their necks
And I leave next to him a message
To children, you show respect

When these corridos report a story such as the case of El M1’s son, it is hard to verify at the time the accuracy of the events described. It is usually until later that people find out what really happened and we confirm whether the corrido was fact or fiction. If this seems like a less than ideal news source, it is. But consider that the same thing can be said about information received from the government (or lack thereof). If the government knew about the intricacies in the rivalries among the cartels that people have heard through narcocorridos, they did not communicate it. Because of government mistrust and censorship in the media, these alternate mediums such as corridos and El Blog del Narco became a very important source of information for people seeking information about the narco-world.

Narcocorridos, altered or unaltered, promote a long list of morally impure practices such as violence, criminal lifestyles, misogyny, homophobia, etc. But they can’t be merely dismissed as such, for they tell story and culture of the last 100 years for a significant part of Mexico. They are complicated and straightforward, deplorable and worthwhile, uninteresting and fascinating. They are a major part of a culture comprised of millions of people, and they can’t be dismissed as merely “songs about violence” because like them or not, they are here to stay.



Ramirez-Pimienta, Juan Carlos. “De Torturaciones, Balas Y Explosiones: Narcocultura, Movimiento Alterado E Hiperrealismo En El Sexenio de Felipe Calderón.” A Contra Corriente 10, no. 3 (Sprin 2013): 302–34.
———. “Del Corrido De Narcotráfico Al Narcocorrido: Orígnes Y Desarrollo Del Canto a Los Traficantes.” Studies in Latin American Popular Culture 23 (2004): 21–41.
Schwarz, Saul. Narco Cultura. Ocean Size Pictures, 2013.
Simonett, Helena. “Narcocorridos: An Emerging Micromusic of Nuevo L. A.” Ethnomusicology 45, no. 2 (Spring-Summer 2001): 315–37.
Wald, Elijah. Narcocorrido:  A Journey into the Music of Drugs, Guns, and Guerrillas. 1st ed. Rayo, 2001.

Impure Hip Hop Dissent

September 7th, 2015

I was reading an essay by Tommie Shelby called “Impure Dissent: Hip Hop and the Political Ethics of Marginalized Black Urban Youth”. In it, Shelby argues that political rap, although it’s lyrics are sometimes misogynistic, homophobic, celebrate violence against cops, and valorize gunplay and street crime, still exhibit an important form of political dissent that should not be cast aside because of its “impurity”. He calls this political rap music “impure dissent”, and argues that it has some intrinsic value since it is not meant to elicit some sort of social change, or somehow change the status quo.

Among the many interesting ideas mentioned, Shelby says that beyond condemning an injustice, impure hip hop dissent has two further functions: “to publicly pledge loyalty to the oppressed, and to explicitly withhold loyalty from the state. {…} This dissent is the expression of solidarity with the oppressed against perceived injustice, not so much because those in power may change course as a result, but because the dissenters want to make clear whose side they are on.”

Another thing that struck me was that while the political speech in hip hop may make its way into the public sphere, it does not necessarily ask for a rational communicative exchange. How could it? When you’re singing about glorifying street crime, you’re not inviting other people to take part in a conversation about its merits. This one-sided dissent may strike other people as “morally impure” since dissenters are refusing to listen much less reply to criticism. “These dissenters … may seem to be lacking in [the] appropriate civic spirit of reciprocity.” Shelby offers another possibility. Dissenters may hold the opinion that critics are arguing in bad faith, and that their unsympathetic attitudes towards their plight is an indication that a meaningful dialog is just not possible.

This reminded me of some of the insensitive responses to the Ferguson protests, in particular the hashtag #PantsUpDontLoot. Perhaps the tiny faction of individuals who were actually involved in the looting could be described as practicing “impure protesting”. They might not have been looking for dialog. Especially not from someone who would reduce the entirety of all racial injustices in the country to “pull up your pants.” Maybe they were looting because they were just really fucking angry.

Shelby’s essay comes from the book “From Voice To Influence: Understanding Citizenship in a Digital Age”, which I highly recommend. It was a happy coincidence that I read this today since I was planning on going to the movies to watch “Straight Outta Compton.”

Movie watching inspired by this sign I saw while walking around Brooklyn today.

Movie watching inspired by this sign I saw while walking around Brooklyn today.

Also: When I thought about writing this I was thinking about how impure hip hop dissent might compare to narcocorridos. Several times I have heard narcocorridos compared to hip hop since they promote and glorify violence. And much like hip hop artists, many narcocorrido artists come from very humble backgrounds and have faced years of neglect and injustice from the state. However, the more I thought about it, the harder it was to make the case that narcocorridos can qualify as impure dissent. Mainly because, in my limited knowledge of narcolyrics, much of that music is not political at all. It mostly just celebrates narcoculture. It’s worth looking into, though. I might also have a very biased opinion since I come from a place that was very much affected by narco violence.

On Evidence-Based Sentencing and the Variables of Race, Age, and Social Achievement

July 25th, 2015

I was reading a paper on evidence-based sentencing called “Risk in Sentencing: Constitutionally-Suspect Variables and Evidence-Based Sentencing“. And in it, the authors list (list is generated by another study), fifteen different variables with statistically significant relationships with recidivism. Here are some (on a 0.30 scale):

  • Criminal companions: z=0.21
  • Antisocial personality: z=0.18
  • Adult criminal history: z=0.17
  • Family rearing practices: z=0.14
  • Social achievement (education, marital status, employment): z=0.13
  • Race: z=0.17
  • Age: z=0.11
  • Gender: z=0.06
  • Socio-economic status of origin: z=0.05

Immediate things that pop out: race, criminal companions (who you hang out with), social achievements (education, marital status, employment), age, gender, and socio-economic status of origin. According to this study, these factors indicate some probability of recidivism. Luckily, several of these variables (such as race and age) are constitutionally barred from being taken into account in sentencing decisions. But the point I want to make is that I don’t think most of these should be a factor in determining a person’s sentence. And I think this study is a great example of why we should be careful when drawing conclusions from analyzing data. I sometimes tell this joke: “100% of divorces are caused by marriage”. It’s silly, but I think it’s relevant here. Yes, divorces begin with marriage, but if you blame marriage on divorce, you’re kind of missing some important underlying cause. Sure, young poor uneducated black people who hang out with other criminals might have an increased chance of recidivism, but is that really the underlying cause? Is it really their fault that they are young, poor, uneducated and black living in a neighborhood where everyone else is young, poor, uneducated, and black?

This is a great example of algorithms just pointing out the obvious and yet missing the larger picture. It’s like Google’s flu detector which actually might only be a winter detector. We need to think about how we construct these algorithms and how we are employing them to make decisions that might affect hundreds of thousands of people. We shouldn’t be asking “how does the race variable relate to recidivism?” There’s nothing “variable” about race. Or age. Or socio-economic status. These are the wrong questions. Instead, why don’t we ask ourselves “What can we do, to improve a person’s life, such that the color of their skin doesn’t correlate with a high recidivism rate?” I think that’s a more worthwhile pursuit.


Book: World Dynamics

July 4th, 2015

I like books. I am not a fast reader. I have not read thousands of books. I don’t even read all the books I own. But I like them nonetheless. I especially like collecting early editions of books that have a special meaning to me or that tried to predict the future, or books that were influential especially in design and technology and how those two fields would or could transform society.

Following Rune Madsen’s lead, I decided to write every now and then about the books I’ve collected. The first one is “World Dynamics” from Jay W. Forrester:

Photo Jul 03, 8 09 47 PM

Forrester was the founder of system dynamics. Unlike the dynamic systems class you might’ve taken in college were you draw bond graphs and model a box-sliding-down-a-ramp-on-a-spring-on-a-damper-on-a-pully-on-a-water-pump-on-a-generator-on-a-moving-train using a system of equations, Forrester’s system dynamics was meant to model complex problems like population growth, use and exhaustion of resources, industrial processes, or determining the success and failures of corporations (the original intent). “World Dynamics” is an application of system dynamics to model the world’s population growth and the exhaustion of resources.

Idea for the study supposedly originated when Forrester met with the founder of Club of Rome, a think-tank that deals with a variety of international issues. Back in the 60’s and early 70’s people had begun to think about the environment, the fear of overpopulation and the exhaustion of resources. This book was meant to give a prediction of what the world could look like in the future. Another book that came out of the Club of Rome was “The Limits to Growth” by Donella Meadows (still trying to get my hands on that one).

The model Forrester produced, predicted that the limiting factor in bringing the world into equilibrium would not only be population growth and the availability of food, but also pollution, crowding, and depletion of resources. Industrialization might be a bigger threat than overpopulation because of the limits of the environment.

The goal of producing the model was to search for an equilibrium where we could sustainably live with earth’s renewable resources. In the search for equilibrium Forrester suggests cutting food production to reduce the population. I’m not sure how exactly he envisioned this playing out, but it seems he was depending on a reduction in birth rates.

Photo Jul 03, 4 51 32 PM



Slightly less complicated than the Afghanistan plan.

This book is one of a long list of books and research that relied heavily on the hope that system dynamics could potentially solve the world’s problems. The idea back then was that if we had all the variables, and we understood how all of them behaved, we might be able to model just about anything, including, literally, the entire world.

And yet, here we are. Still unable to solve the world’s problems, having long abandoned the idea that we can model the world simply by coming up with the right equations.


Verizon’s Morse Code Post, Translated

February 26th, 2015

Today the FCC ruled in favor of Net Neutrality. Opponents such as Verizon, thought this was an antiquated decision. So in what is probably the most childish response by a corporation I’ve seen to date, Verizon responded with a blog post stating their disappointment at the FCC’s decision. In Morse Code. They also provided a link to a PDF. With typewriter-style-smudged-ink text. Dated Feb. 26, 1934.

All right Verizon, challenge accepted.

I copy-pasted their blog post and ran it through a short python script I made. This is what comes out (formatting mine).

“Today’s decision by the FCC to encumber broadband internet services with badly antiquated regulations is a radical step that presages a time of uncertainty for consumers, innovators and investors. Over the past two decades a bipartisan, light-touch policy approach unleashed unprecedented investment and enabled the broadband internet age consumers now enjoy. The FCC today chose to change the way the commercial internet has operated since its creation. Changing a platform that has been so successful should be done, if at all, only after careful policy analysis, full transparency, and by the legislature, which is constitutionally charged with determining policy. As a result, it is likely that history will judge today’s actions as misguided. The FCC’s move is especially regrettable because it is wholly unnecessary. The FCC had targeted tools available to preserve an open internet, but instead chose to use this order as an excuse to adopt 300-plus pages of broad and open-ended regulatory arcana that will have unintended negative consequences for consumers and various parts of the internet ecosystem for years to come. What has been and will remain constant before, during and after the existence of any regulations is Verizon’s commitment to an open internet that provides consumers with competitive broadband choices and internet access when, where, and how they want.”

“Verizon’s commitment to an open internet that provides consumers with competitive broadband choices” Really, Verizon? Really?

The Mexican GovBots Did NOT Take Down #YaMeCanse, But We Can Keep #YaMeCanse# Trending

December 8th, 2014

Perhaps it was the excitement of hearing about a new phenomenon in censorship that prompted me to write a little too hastily about how the government of Mexico might have used Twitter bots to spam and trash the #YaMeCanse hashtag out of the trending topics list. As reported by Lo Que Sigue,, Aristegui Noticias, and myself, #YaMeCanse, the hashtag used as the rallying cry for Mexico’s 43 missing students, was suddenly dropped from the trending topic list by an army of bots, presumably coordinated by the federal government. No proof is provided by any of us that the government was behind this, but a series of videos and screenshots originally provided by Lo Que Sigue lead us to believe that a swarm of bots is at least responsible.

It was NOT the bots

December 3 at 10:36 AM was the last time @TrendieMX reported #YaMeCanse to be trending. By 9:36 PM of the next day, #YaMeCanse2 was already trending. Let’s take a look at what the Topsy trends for #YaMeCanse looks like for the month of November and the first days of December.

Usage of #YaMeCanse


To make sense of what happened, we need to understand what Twitter is doing to calculate a trending topic. We don’t have access to specific information about how the trending algorithm functions, but we do know how trending algorithms in work general and we have some clues about what Twitter has done in the past to tweak its algorithms. The relevant issue here can be described as “the Justin Bieber problem”. Many of you might remember how some years ago Justin Bieber was constantly trending due to the millions of Beliebers continuously tweeting about him. Twitter wants to tell us what’s trending right now, and not one hour ago or one month ago. As Twitter is quoted saying in this Mashable article:

“The new algorithm identifies topics that are immediately popular, rather than topics that have been popular for a while or on a daily basis, to help people discover the ‘most breaking’ breaking news from across the world. (We had previously built in this ’emergent’ algorithm for all local trends, described below.) We think that trending topics which capture the hottest emerging trends and topics of discussion on Twitter are the most interesting.”

Instead of merely looking at volume of Bieber tweets (of which there are many), Twitter looks at speed and “burstiness” of the tweets. However, there’s more to it. If Twitter only measured “burstiness”, you might see “Good Morning” trending every morning of every day. For this, Twitter establishes a baseline of expected frequencies based on history. Twitter “knows” there is usually a spike of “Good Morning” tweets every morning and corrects for this. As this video on trend detection in twitter social data explains, a ratio is calculated for each term based on the past frequency of the term and the present frequency.

What most likely happened is that after a couple of weeks of trending, the baseline for #YaMeCanse rose from zero (it didn’t exist before 11/7) to the frequency of people tweeting at the end of November. Twitter treated the volume and speed of the hashtag as something it would expect and dropped it off the trending list.

Baseline shift on #YaMeCanse

Spam bots should have no impact on the algorithm. The spam team at Twitter identifies the bots and they are not counted towards the algorithm. Additionally, it’s worth mentioning that Twitter has a team of low-paid human workers manually sorting through hashtags to eliminate advertiser spam. Even so, there is no evidence of an increase in bots during the time the hashtag was dropped from the list. The team at Lo Que Sea provided this video as proof of the presence of bots (not that we need proof of that in general)

Screenshot of Lo Que Sea video

Why are individual and unconnected tweets labeled as bots? If I tweet and only one person RT’s me, by their standards, I’m a bot. You can run the simulation from the video yourself on Use #YaMeCanse2 and wait for the same pattern of connected and disconnected tweets to occur. Then zoom in on the disconnected tweets and look up a couple of usernames. You’ll find a lot of those disconnected nodes are real people. You’ll also run into bots, but having no one retweet your tweet does not make you a bot.

This has happened before.

This would not be the first time that people have cried censorship upon the disappearance of a hashtag from the TT list. The Mashable article quoted above was a response to Beliebers accusing Twitter of censorship. Similarly, occupiers accused Twitter of censorship when #OccupyWallStreet was taken off the list. In both occasions Twitter had to step in and say this was just a result of how the algorithm works. In some cases we should be glad the algorithm works like this, otherwise we’d see #JustinBieber constantly trending. But how about when it’s something important like #YaMeCanse?

At this point I should say that if it were possible for the Mexican government to use such a tactic to censor people on social media, they probably would. We’ve already seen how Peña Nieto’s campaign used bots to promote the candidate on Twitter. And earlier this year, an initiative put forth by Peña Nieto on Radio and Telecom caused a lot of controversy when people claimed the law would allow the government to censor online content and to interrupt cell reception during protests. There’s also the case of which was censored by GoDaddy under pressure from the US Consulate in Mexico.

We’ve Discovered How To Get Around the Algorithm

I believe the immediate response by the Mexican Twitterverse in the creation of #YaMeCanse2 has revealed an exploitable feature in the algorithm. It took less than two days for people to adapt to the new hashtag. Topsy Trends shows that #YaMeCanse2 doesn’t have significantly more traffic than #YaMeCanse had before being taken down, but the reason why #YaMeCanse2 was able to trend so quickly is because its baseline at the time was zero. This means whenever #YaMeCanse2’s baseline shifts up enough for it to de-trend, we can just start again with #YaMeCanse3. We can keep going with this as long as we keep the speed at which people tweet constant or as long as Twitter doesn’t catch on and modifies the algorithm to account for us just adding a number at the end of the phrase (In which case we can just add the word “tres”). This is also why we keep seeing so many different Bieber hashtags, they’re all different phrases that didn’t exist before.

The lesson here for the Mexican folk is that if we want to continue to have a #YaMeCanse hashtag trending, we need to coordinate to increment the number at the end of the tag each time it expires. When #YaMeCanse2 falls off the list, we simply switch over to #YaMeCanse3.

Censorship on Twitter Using Bots? How #YaMeCanse Was Knocked Off Twitter Trending Topics

December 4th, 2014


In late September of this year, 43 students in the Mexican state of Guerrero went missing. In an attempt to prevent students from disrupting a political event for his wife, the mayor of Iguala ordered local police to stop and detain the students. This set in motion a series of events that resulted in several murdered students and 43 missing students. People later learned the missing students were handed over to a local cartel and were subsequently killed and burned until no traces of their bodies were left behind. This announcement was made during a press conference by Mexico’s Attorney General, Jesús Murillo Karam where at the end of the conference, tired and exasperated, said “Ya me cansé.” I’ve had enough.


Mexicans took to social media and responded with “We’re tired too…” Of the violence. Of the injustice. Of the impunity. Of the corruption. The #YaMeCanse hashtag became the rallying cry for discourse online and protests all over Mexico. The hashtag has been on Twitter trending topics almost since Murillo Karam’s press conference. Yesterday, the hashtag suddenly disappeared from the list even though usage had not waned.

Usage of #YaMeCanse


This sudden disappearance of such a popular hashtag raised some eyebrows. Determining trending topics is a little more complicated than simply calculating the number of mentions of a hashtag. Twitter has an algorithm that determines trending topics based on several factors. According to Twitter, one of the rules against usage of trending topics is “Repeatedly Tweeting the same topic/hashtag without adding value to the conversation in an attempt to get the topic trending or trending higher.” It is very likely that the overwhelming spamming of the #YaMeCanse caused Twitter’s algorithms to treat the hashtag as spam and proceded to remove it from the trending list.

As reported in, an army of bots had been RT and tweeting the #YaMeCanse hashtag for several days.

“Who says that online censorship and repression does not exist online? A storm of bots tries to disappear #YaMeCanse”

Spam Tweets

Another analysis by Lo Que Sigue shows the difference between connected and disconnected tweets symbolizing real people versus bots.


Not to be easily dissuaded, the Mexican twitterverse quickly came up with a simple solution: #YaMeCanse2, which is currently trending. An added cleverness to adding the number ‘2’ is that it forces people to ask “What happened to regular #YaMeCanse? Where’s #YaMeCanse1?” which leads people to find out about the attack. It’s a sort of the Barbra Streissand effect where in an attempt to censor one hashtag, not only do people evade the censorship, but in doing so call attention to the attempt at censorship.

It’s quite possible that this is not a coordinated attack on the hashtag by some entity. It could be just regular bots hijacking a popular hashtag. And it is very tempting to attribute to this “attack” to the government of Mexico. I would not be surprised at all if it was, and I’d be willing to bet that the Mexican government is behind this (it wouldn’t be the first time), but I would like to find definitive proof. The people behind Lo Que Sigue working to start an Indiegogo campaign to try and find the origin of these bots. Perhaps we don’t have to wait around for this to get funded and we can crowdsource/collaborate to try and see if tracing the origin of the bots is possible. I would welcome any ideas on how to do this.


So Trending Topics are more complicated than they seem. It’s hard to tell whether bots had any role in dropping the hashtag from the trending list. It seems that Twitter is actually looking for “bursts” of tweets, and how fast these tweets appear ( ∂Tw/∂t?). It is entirely possible that volume of tweets remained stable but the “burstiness” was gone. I don’t know. Twitter’s algorithms are very private. Even if bots played no part in dropping the hashtag, the possibility of that happening might still exist. After all, riding hashtags to promote unrelated content is shunned by Twitter. Whether they can detect that algorithmically, I’m not sure, but I wouldn’t be surprised. If they can detect that, then it’s entirely possible to spam a hashtag using bots. Perhaps the only way to find out is to actually measure the volume and speed of the bots. Doing this, it turns out, is very hard.


“Social Physics: How Good Ideas Spread–The Lessons From a New Science” – Alex Pentland

November 10th, 2014

I started reading Alex “Sandy” Pentland’s book, Social Physics. Several things interest me about this book. I’m very interested in how society behaves in today’s world where we are increasingly connected to more people by weak social ties. Also interesting is that advances in data collection and analysis are bound to reach a point where we can continuously monitor and analyze people’s behavior. Who will have this knowledge? How will they use it? What will this world look like? Lastly, I’m interested on how good ideas spread and how that can help us design better organizations and institutions.

Alex Pentland thinks it is possible to create a mathematical explanation about why society behaves the way it does. He calls this discipline, social physics.

“Social physics is a quantitative social science that describes reliable, mathematical connections between information and idea flow on the one hand and people’s behavior on the other. Social physics helps us understand how ideas flow from person to person through the mechanism of social learning and how this flow of ideas ends up shaping the norms, productivity, and creative output of our companies, cities, and societies. “

The goal of applying this science to society is to shape outcomes. Pentland believes we can create systems that build a society “better at avoiding market crashes, ethnic and religious violence, political stalemates, widespread corruption, and dangerous concentration of power.”

All of this would sound great, if it didn’t sound kind of scary. There are a lot of concerns about privacy, which Pentland addresses, and which I’m sure he’ll talk more about in the coming chapters. However, even if he is able to get around the privacy issues, the ability to affect how society behaves would give whoever has the ability to do so great power. This is perhaps a little paranoid on my part, but I don’t think misusing the ability to “fix” society, as he puts it, is out of the question. Pentland does write about it:

“This vision of a data-driven society implicitly assumes the data will not be abused. The ability to see the details of the market, political revolutions, and to be able to predict and control them is a case of Promethean fire—it could be used for good or for ill.”

My second concern is best summarized by Nicholas Carr in his article in “The Limits of Social Engineering”.

“Pentland may be right that our behavior is determined largely by social norms and the influences of our peers, but what he fails to see is that those norms and influences are themselves shaped by history, politics, and economics, not to mention power and prejudice. People don’t have complete freedom in choosing their peer groups. Their choices are constrained by where they live, where they come from, how much money they have, and what they look like. A statistical model of society that ignores issues of class, that takes patterns of influence as givens rather than as historical contingencies, will tend to perpetuate existing social structures and dynamics. It will encourage us to optimize the status quo rather than challenge it.” (h/t to Cathy O’Neil for linking to this piece).

The case studies in the book so far take place in groups where this might not be a huge issue like eToro, an online trading and investment network. Carr’s (and my) concern may not be a huge issue in these scenarios especially because Pentland is measuring very specific metrics like return on investments. However I do believe there is real danger in applying this sort of analyses in places like, say Ferguson, MO. It will be interesting to read the different case studies and to try and identify places where this concern might arise.

It would be very unfair of me to end this without writing about the actual focus of the book (although I’m already a little nauseous fro writing this on the train). The book will focus on the two most important concepts of social physics: idea flow within social networks and social learning, that is, how we take these new ideas and turn them into habit and how learning can be accelerated and shaped by social pressure.

I like to believe that there are better systems of collaboration and cooperation that can make organizations more effective, communities more resilient, and authorities more accountable. Elinor Ostrom developed her work on governing the commons by studying how communities behaved around issues like irrigation and water management. Similarly, I do think Pentland’s insights on idea flow and social learning can help us understand how to design better organizations, communities, and institutions.

The Dangers of Evidence-Based Sentencing

October 27th, 2014
Note: This post was originally published on and cross-posted on

What is Evidence-based Sentencing?

For several decades, parole and probation departments have been using research-backed assessments to determine the best supervision and treatment strategies for offenders to try and reduce the risk of recidivism. In recent years, state and county justice systems have started to apply these risk and needs assessment tools (RNA’s) to other parts of the criminal process.

Of particular concern is the use of automated tools to determine imprisonment terms. This relatively new practice of applying RNA information into the sentencing process is known as evidence-based sentencing (EBS).

What the Models Do

The different parameters used to determine risk vary by state, and most EBS tools use information that has been central to sentencing schemes for many years such as an offender’s criminal history. However, an increasing amount of states have been utilizing static factors such as gender, age, marital status, education level, employment history, and other demographic information to determine risk and inform sentencing. Especially alarming is the fact that the majority of these risk assessment tools do not take an offender’s particular case into account.

This practice has drawn sharp criticism from Attorney General Eric Holder who says “using static factors from a criminal’s background could perpetuate racial bias in a system that already delivers 20% longer sentences for young black men than for other offenders.” In the annual letter to the US Sentencing Commission, the Attorney General’s Office states that “utilizing such tools for determining prison sentences to be served will have a disparate and adverse impact on offenders from poor communities already struggling with social ills.” Other concerns cite the probable unconstitutionality of using group-based characteristics in risk assessments.

Where the Models Are Used

It is difficult to precisely quantify how many states and counties currently implement these instruments, although at least 20 states have implemented some form of EBS. Some of the states or states with counties that have implemented some sort of EBS (any type of sentencing: parole, imprisonment, etc) are: Pennsylvania, Tennessee, Vermont, Kentucky, Virginia, ArizonaColorado, California, Idaho, Indiana, Missouri, Nebraska, Ohio, Oregon, Texas, and Wisconsin.

The Role of Race, Education, and Friendship

Overwhelmingly states do not include race in the risk assessments since there seems to be a general consensus that doing so would be unconstitutional. However, even though these tools do not take race into consideration directly, many of the variables used such as economic status, education level, and employment correlate with race. African-Americans and Hispanics are already disproportionately incarcerated and determining sentences based on these variables might cause further racial disparities.

The very socioeconomic characteristics such as income and education level used in risk assessments are the characteristics that are already strong predictors of whether someone will go to prison. For example, high school dropouts are 47 times more likely to be incarcerated than people in their similar age group who received a four-year college degree. It is reasonable to suspect that courts that include education level as a risk predictor will further exacerbate thesedisparities.

Some states, such as Texas, take into account peer relations and considers associating with other offenders as a “salient problem”. Considering that Texas is in 4th place in the rate of people under some sort of correctional control (parole, probation, etc) and that the rate is 1 in 11 for black males in the United States it is likely that this metric would disproportionately affect African-Americans.

Sonja Starr’s paper

Even so, in some cases, socioeconomic and demographic variables receive significant weight. In her forthcoming paper in the Stanford Law Review, Sonja Starr provides a telling example of how these factors are used in presentence reports. From her paper:

For instance, in Missouri, pre-sentence reports include a score for each defendant on a scale from -8 to 7, where “4-7 is rated ‘good,’ 2-3 is ‘above average,’ 0-1 is ‘average’, -1 to -2 is ‘below average,’ and -3 to -8 is ‘poor.’ Unlike most instruments in use, Missouri’s does not include gender. However, an unemployed high school dropout will score three points worse than an employed high school graduate—potentially making the difference between “good” and “average,” or between “average” and “poor.” Likewise, a defendant under age 22 will score three points worse than a defendant over 45. By comparison, having previously served time in prison is worth one point; having four or more prior misdemeanor convictions that resulted in jail time adds one point (three or fewer adds none); having previously had parole or probation revoked is worth one point; and a prison escape is worth one point. Meanwhile, current crime type and severity receive no weight.

Starr argues that such simple point systems may “linearize” a variable’s effect. In the underlying regression models used to calculate risk, some of the variable’s effects do not translate linearly into changes in probability of recidivism, but they are treated as such by the model.

Another criticism Starr makes is that they often make predictions on an individual based on averages of a group. Starr says these predictions can predict with reasonable precision the average recidivism rate for all offenders who share the same characteristics as the defendant, but that does not make it necessarily useful for individual predictions.

The Future of EBS Tools

The Model Penal Code is currently in the process of being revised and is set to include these risk assessment tools in the sentencing process. According to Starr, this is a serious development because it reflects the increased support of these practices and because of the Model Penal Code’s great influence in guiding penal codes in other states. Attorney General Eric Holder has already spoken against the practice, but it will be interesting to see whether his successor will continue this campaign.

Even if EBS can accurately measure risk of recidivism (which is uncertain according to Starr), does that mean that a greater prison sentence will result in less future offenses after the offender is released? EBS does not seek to answer this question. Further, if knowing there is a harsh penalty for a particular crime is a deterrent to commit said crime, wouldn’t adding more uncertainty to sentencing (EBS tools are not always transparent and sometimes proprietary) effectively remove this deterrent?

Even though many questions remain unanswered and while several people have been critical of the practice, it seems like there is great support for the use of these instruments. They are especially easy to support when they are overwhelmingly regarded as progressive and scientific, something Starr refutes. While there is certainly a place for data analytics and actuarial methods in the criminal justice system, it is important that such research be applied with the appropriate caution. Or perhaps not at all. Even if the tools had full statistical support, the risk of further exacerbating an already disparate criminal justice system should be enough to halt this practice.

Both Starr and Holder believe there is a strong case to be made that the risk prediction instruments now in use are unconstitutional. But EBS has strong advocates, so it’s a difficult subject. Ultimately, evidence-based sentencing is used to determine a person’s sentencing not based on what the person has done, but who that person is.