Taming the swarm

school of fishThe continuous attempts to tame the flood of information have a lot in common with detecting emergent patterns in complex systems: we look for a deterministic model that can explain the behavior detected in the large number of observations, present that model and observe how it stacks out against further observation that might or might not fit into the model.

Our theoretical thinking, though, is limited to the outliers that actually perform the creation of the models, the examination of the underlying assumptions and the critical review of the results presented.

Yes, the large majority of data scientists behave in a critical manner, explore within the limits of their modelling tools, and then use those limitations to explore further, refine and limit their model and observation parameters.

Those limitations, the constraints inherent to the production of a theory of the particular model under study and the limitations of imperfect knowledge and limited computing time move scientists to accept limited modeling techniques, as well as try to extend the metaphor of the underlying theory, effectively assuming the model used to study physical or economic events might use the limited assumptions of theoretical spaces within which the model is first derived.

For example, the linear model attempting to simulate and predict a situation in which there are at least four independent variables is going to fall short of explaining situations that arise in those variables, Furthermore, the assumption that the model is linear is also one that further reduces the validity of said model, since it is impossible to explain various nonlinear effects that might occur. The presence of higher order components will lead to interesting behavior that can be explained with many different mathematical tools, but definitely not within the framework of a first order linear model.

Seeing this happen in academia makes me think about the silo nature of research: models that could benefit from the hand of a mathematician or physicist are left unregarded, barely explained, while extra rigour is demanded of the most simple of models, in an attempt tom formalize and eliminate criticism due to methodology, when that methodology is not at fault here: the initial assumptions forget that the reality being described is perhaps better described using some other, higher order, dynamic description.

Meanwhile , we get to listen as migration is explained as dependent on happiness, while a whole other raft of environmental, economic and social factors are left unmentioned.

Posted in Academia, BI

Another Fourth

These days that handle heat and patriotism, mixed with bubbles, street carnivals and overpriced flags made in China, are also reminders of subtler emotions, deeds and promises: we all read the books, and can almost TH understand the meaning and connect with the context, but history evades us amid the myriad distractions posted by the press, the friends and the parties. 4th of parties indeed, because we all became engrossed in enjoying a few days off, relaxing and for a couple days, allowing time to flow without a particular goal but that of letting the sun go down.

Simmer in the heat, allow the chair to creak and the ice melt, high pitched noises in the background squeals of delight and enjoyment, all along with the threat of storms and bugs happily ignored.

Tired feet, air conditioners on high, ice cream for once being a valid option, and a long walk home after a town festival, caring our thoughts about independence and rights as some cherished antiquities that inform our present hey need to be asked and aired far more often.

Posted in Chronicles from The Island, Uncategorized

In the South and West, a Tax on Being Poor

tax dayEverywhere we see that Wilkinson’s book The Spirit Level applies; one of these studies points to regressive taxes as indicators of inequality and their higher associated costs: The NYT published an article about it: In the South and West, a Tax on Being Poor, and one of the paragraphs sums it succinctly:

For every $100 increase on taxes at the poverty line, we saw an additional 7 deaths and 78 property crimes per 100,000 people, and a quarter of a percentage point decrease in high school completion.

The taxes tell part of the story, but as with other indicators, they are only a glimpse into a system that skews towards affluent minority, while eliminating the social safety net and penalizing the lower percentiles in ways that eliminate mobility, increase morbidity and places additional costs on the social group.

Early on we were having discussion on the social contract, and how the perception of that contract allows people, constituents, to express their concerns and have actual results in a decent time frame, thus diminishing the pain that misguided policies might inflict. However, when the contract emphasizes the individual right over the social one, we see issues such as the ones presented in the NYT article.

Austerity has long proven to be an ineffectual tool, and the policies resulting on measures that resemble that are painful, regressive and costly.

Posted in Economics, Politics

Promotion vs. Prevention

time machine
And now it comes down to how the PM directs their team: Either by focusing on the final goal, identifying possibilities and taking risks in attaining that goal, or by making sure that the product managing process is taken care of, that the stakeholders all maintain proper communication through channels, and that all figures related to the development and testing process are in place.

HBR now presents that, depending on the manager focus, a product manager might too focused on a promotion or prevention style; it could easily be argued that a promotion focus serves better the startup PM, but it has also been shown that a prevention focus is particularly used by startups when dealing with the later stages of the product management cycle.

Quiz: What’s your style?

Article

Do You Play to Win — or to Not Lose?.

Tagged with: , ,
Posted in Business, Management, Product

On perceptions and paradigms as constraints

rails in de sneeuw
Good literature has this habit of turning assumptions on its head, explaining the limitations of perceptions and the ways in which we, as humans, maintain artificial sets of constraints even though we might be trying to fight those same obstacles. Pretty much the way Arendt was criticized, we lose our product vision, because precisely the way we have to fight the forces that push seemingly innocuous agendas on our plate. We have to regroup, reassess and act again.

These assumptions that so easily maintain the status quo, or impede the realization of our initial vision, also affect us, derailing our intentions and helping us, and others, into rationalizing the very same obstacles into being necessary.

For example, a process that has been unreliable, or a continuous rechecking and retesting, or a routine series of tasks that take resources yet are performed because that is the way that it has been done before, or because they are necessary to maintain data integrity on legacy systems, all seem to be valid uses of the time of developers and managers. The problem starts with reason for those routine tests and tasks in the first place: is the system getting into legacy mode? Are the input processes correct and updated? Is there a way to fix all these issues that pop up under use, and while the system is stress tested by changing business demands and requirements?

There is no correct answer here: the business evolves and changes requirements, the installed system becomes legacy, and resources and allocation are scarce; the legacy IT system is already established, the implementation of another system is dependent on business timing, i.e. you won’t change Point of Sale just before Christmas season, and recovering the existing knowledge to incorporate it into a new application is serious enough – old, legacy work is knowable, routine and easily measured, although prone to errors and mistakes.
And that is where the vision vanishes, replaced by routine task management and endless quality control tasks that are precisely what has launched the new system implementation in the first place.

But who manages that implementation, and how does that proceed?

Posted in Management, Product

Another SaaS BI. We told you

And now techchrucnch has that Domo is getting new funding.

The tech space has always been awash with things looking to give to the exec suite, especially since they are the ones with the pourse strings and the Damocle’s sword hanging over them. They want results now, ubt they want information and insight yesterday!

It is really discomfiting that an F500 would say

the company had to wait months to analyze sales data and then make changes in other business areas

Really? Don’t you, big F500, have some BI already and process all your sales data internally?

Well, apparently not. I am wathcing other potential SaaS BI like Birst, but it seems clear that Domo wnats to differentiate itself by virtue of its target market and its final users.

Posted in BI, Business

Medicaid Stance

The discussion on implementation of ACA by states gets heated, but I find it worthy of attention the map showing the states that are going ahead with an implementation of the Act, and how it correlates to the infamous “right to work” states.


Where the States Stand

Via: The Advisory Board Company

One: We need a cartograph showing population affected.
Two: Costs to the nation as a whole, because of the reticence of some politicized states to adopt federal law?
Three: Costs to society in the medium term as a result of the botched implementation of this law, specially shown by state: growth, economic puissance, local CPI?

Posted in Economics, Politics

Twitter Weekly Updates for 2012-10-14

Powered by Twitter Tools

Posted in Twitter Post

Twitter Weekly Updates for 2012-10-07

Powered by Twitter Tools

Posted in Twitter Post

Seeing #47percent in Gephi

47PerMain.pngNow that we have debates coming in, let’s see what Twitter had to say about the 47 percent comment from a while ago.
As you can see from the graph, there were a few major players twittering that thing, with 3 of them echoing the MotherJones sentiment, while the other one was BlackRepublican, agreeing with Willard and tryng to make the hashtag #47percent a rallying cry.

While you saw a lot of people decrying the comment made on the infamous video, a lot of people that vote Republican simply saw that as an affirmation of where their candidate sits, and interpreted that comment as a good thing.

As we all live in a social media echo chamber, it is necessary to use other tools to see what the general perception of the society is; particularly in this case, when groups see what they want to see and are quick to dismiss other alternative points of view or competing explanations.

Of course, this contrary explanation that we are seeing for the interpretation of the 47 percent video occurs because we have a polarized debate, one in which things are either right or wrong but nothing in between: We have to change that debate strategy, and find other ways to involve and listen to the other side.

Tools used:L Twitter, Gephi, RetweetMonitor.

Posted in Networks, Politics, Social Networks, Web2.0