Friday, October 24, 2014

SENECA FSOSS 2014 REPORT

Abstract

Comparing and analyzing two symposium talks from industry leaders of open source technology in the world of IT.

“How Companies Use NoSQL Open-Source Technologies like Couchbase” – Don Pinto, Product Marketing Manager (Couchbase)

Don Pinto (M.Sc., Computer Science – University of Toronto) has previously worked as the director of product management at GridCentric Inc. (now owned by Google), with additional experience as a SQL Server/Azure program manager at Microsoft.

The Problem – “There is lots and lots of data. More users than ever before and the interactive complexity of apps.”

“Consumers & Employees Demand Highly Responsive Apps.”

Old relational stores had a lack of flexibility/rigid and an inability to scale out data easily. Those 2 factors along with performance costs comprised of some of the most popular client complaints when using such tools.

This calls for a new backend technology – NoSQL.
So what could be a candidate to be the right tool?

·        The JSON Data Model Fits today’s developer needs better
o   Aggregates & denormalizes data into single document (Document data model).
o   Handles structured & unstructured data equally well (Docs are distributed evenly across servers)
o   Inferred schema requires no migration
o   JSON rapidly being adopted
o   Access both JSON and binary data as key-value pairs

·        RDBMS needs a bigger, more expensive server to scale up architecture.
·        Auto-sharding vs. Manual sharding (data partitioning).
·        Open-source obviously implies lower costs for maintenance, and usage.

·        Availability – Relational systems use clustering as an afterthought.
o   RDBMS must take database down for “maintenance windows”
o   They struggle to support XDCR (Cross data center replication) across many DCs (data centers).

·        Couchbase offers a full range of Data Management solutions
o   High Availability Cache (Zero downtime administration and upgrades)
§  Always-on functionality for a potentially global user base
§  Couchbase Lite – Mobile application that includes a sync gateway for mobile work to update server.
o   Consistet High Performance
§  Built-in object level cache
§  Fine grained locking
§  Hash partitioning to uniformly distribute data across the cluster
o   Elastic Scalability –
§  Shared-nothing architecture with a single node type
§  True XDCR
§  Push button scale-out

Some use cases for NoSQL:
·        Heavily accessed web landing pages
·        Application objects
·        Popular search query results
·        Session values or cookies (key-value pair store), eg. Shopping carts, flights selected, etc.
·        User profile with a unique ID, user settings/preferences, user application state
·        Content metadata stores (articles, text)

Some known users of Couchbase:
·        Orbitz – 11 clusters with a total of 100 nodes
o   3 TB of data with over 430 million objects
·        McGaw-Hill Education Labs – Content and metadata stores
o   “Building a self-adapting, interactive learning portal”
o   Scale to millions of learners
o   Self-adapt via usage data
·        AOL – Ad-targeting using a Couchbase server
o   40 milliseconds to respond with the decision.
o   User profiles, real time campaign stats
o   Affiliate, event, profile, and campaign data

Mike Hoye, Engineer Community Manager at Mozilla: “Social Engineering – Building Communities With, And On, Purpose”

·       "Process reifies and reinforces values"
·        If you don’t measure it, don’t pretend you care about it
·        The ROI on timely gratitude is ridiculous.
·        Karma is a wheel (courtesy, saying thank you for the things you are given)

“The way you conduct and execute your process is a direct reflection of your values.”

Access, Engagement, Retention - “If you let a patch sit for a week from a first contributor, it is very unlikely you will see them contribute again.”

Accessibility
·        What is in front of a user, if they want to commit a one-line change to your project?
·        The importance of comprehensive documentation
·        The “miraculous” benefits of an easy-to-set-up build environment

Engagement
·         * “Throw the little fish back in the water for the new entrants to the game.”
·         *  Label good first bugs for beginner contributors and give a concise, yet thorough explanation of how to go about fixing them

Retention
·         * “A single toxic contributor can harm an entire community. If people feel unequally welcome in the     community, many will inevitably shy away from it. Don’t be a jerk and don’t let others become           jerks.”
        * Gratitude. Saying thank you. “This bug and your fix matters.” Telling them what to do next.

Why does open source matter? Am I the first person to have this problem?

-        Mythology: Is what is in the absence of real numbers and real data about what works and what doesn’t. Stories are powerful and get into and stay in people’s heads.

-        Open source is meritocratic (we need to stop talking about ourselves like we’re “magic”)

-        Diminishing returns: After 3 sets of eyes looking at a piece of code to figure out a bug, the rest are wasting their time…

-        Strong FSOSS and FSOSS-like communities grow organically

What are the most basic, fundamental things we need to embark upon an open source project?

1)      Source Control
2)      Issue Tracking
3)      Automatic Testing

Do you care?
“There is no regression test for somebody’s mood.”

“Your community is an API to your software.”
What is the state of our community? What problems does our community have?
Are we actively fostering community engagement?

“Have a code of conduct. Have a code of conduct. Have a code of conduct.”

               Choosing this particular pair of talks yielded a very comprehensive picture on the open-source process, since one focused on the community build and project goal philosophies – the preliminary prerequisites - and the other presented the benefits of a completed and deployed product that was a result of many of the same ideals and principles being implemented. There was a clear testament to not only an implied agreement of the two speakers’ points and values, but a symbiosis as well. The focuses and comparisons were of a completely different nature and centered on relatively unrelated processes due to the subject matter, but were ultimately two sides of the same coin.

The open-source paradigm and its implications have not changed for me so much as they had flourished over time. Admiring the philosophies behind open source came naturally, but I was initially confused and doubtful about the financial feasibility of companies and institutions that fully embrace free software and open source processes and values with their intellectual property. The state of this maturing industry after nearly a quarter-century of existence is clear evidence of its resounding and continued success, mainly by awareness of giants like Microsoft or Google investing in more open source venture startups and releasing more open source code and products, as well as witnessing and contributing to companies like RedHat and Mozilla - which are almost entirely based off of open source ideologies in every one of its aspects - rise over the years to become fortune 500 companies.

Thursday, October 16, 2014

0.2 Milestone Completed - My 1st Appmaker Pull Request is in!

First and foremost, the link to the PR - Fixed tray hover tooltip position offset #2317

As expected, this bug ended up being nothing short of an absolute blessing because the time it took to fix it was an iota of the amount of time it took to successfully set up my Appmaker development environment on my Windows 8 machine in order to be able to properly see and test my work.

In the beginning, besides the odd npm install error log emitting after attempting to download the required dependencies partially failed raising an eyebrow, it was relatively smooth sailing until needing to install and run MongoDB. The Appmaker documentation doesn't really let on much as far as the nitty gritty to my particular use case, save for showing the general link to the official website tutorial for installation and use steps. Those proved to be little more than useless in the end. I GUI-installed MongoDB just fine, but as soon as I opened up a command prompt to try and run it as per the instructions on the Appmaker README it would fail and often close the window altogether. When I managed to isolate the error it purported, it looked something like this:

********************************************************************
 ERROR: dbpath (\data\db\) does not exist.
 Create this directory or give existing directory in --dbpath.
 See http://dochub.mongodb.org/core/startingandstoppingmongo
********************************************************************

But much like Tweety Bird constantly reaffirming that it did, it did in fact see a pootie tat... I too clearly made a \data\db folder and later even set and assigned that location to the dbpath explicitly in the config file. After trying to make the most of my google-fu skills to find a step-by-step how-to on Windows related MongoDB installation and execution, I finally found the most relevant and comprehensive one titled Running MongoDB on Windows. If you follow this with a fine-tooth comb, you will be able to both start mongo on both the command prompt, and have it perpetually keep running as a local service via the service manager.

NOTE: For the love of all that is good and righteous in this world, the Service Controller (sc.exe) command you will try to run to create a local service out of MongoDB can ONLY be executed by the administrator account -- not your own custom user account, even if it has the "administrator" label/type. If you want to be extremely thorough, follow the steps mentioned in this answer article to the same problem for a different sc command.

--------------------------------------------------------------------------------------------------------------------------

Finally, the bug itself proved to be initially daunting, but with inspiration from a casual comment given to me in-class on Wednesday from a fellow student, once I managed to get the node app/server to run, the procedure I followed ended up being close to my original strategy with a few additions:

1) Open my Appmaker localhost browser instance.

2) Find the "tray hover tooltip" element as described in the bug. It wasn't called that, so after thinking about it, I actually just used my own vision to infer where its location on the DOM was by looking at the image posted on the issue site in Github and compared it with my Appmaker site. I know, it's horribly crass, but hey, it seems to have worked out this time...

3) I used the Chrome Developer Tools to try and inspect the element by hovering over it. This was where my biggest issue was - the tray hover tooltip only appears when you hover over the tray's inquiry icon. Because life was never meant to be this easy. After a bit of thought, I figure I might be able to pinpoint it in the source code by finding where the logic for its parent element was - a div with an ID named "showInfo".

4) In Atom, I initiated a global search for that particular string. I ended up finding it in a place I would never have guessed from past experience with css properties - an .html file. What? Fine, I'll indulge in new flavors. So I started parsing the rest of the file and ended up landing on this interesting looking property nugget:

floatingMoreInfoPopup.style.top = pos + 54 - (height/2) - offset + "px";

I'll be honest. I don't have any background context on the specifics of this statement. It looks more like Javascript than CSS so my assumption is that it's some sort of new library or framework for CSS injection that I'm not yet aware of, or something much more comprehensive and cool altogether. What I did see is the name of the property itself, which by the looks of it seemed like just about exactly what I was looking for. So I did what any Curious Carl would do at this point...

5) Start playing around with values, and see what changes. Turned out to be pretty much as easy as that. I changed the pixel value after the "pos" variable by 14 pixels and voila!

The original positioning:


My fix's positioning:


And for now, that's all she wrote ladies and gentlemen. Stay tuned for next week update on FSOSS!

Wednesday, October 15, 2014

0.2 Milestone Progress Report

A few weeks into the workload at hand, and after finding a bug and initially attempting to setup the development environment a few times, a problem that seems to be repeating itself in recent times is the inherent constraint of other work from full-time courses as well as a cursed history of personal less-than-stellar time management skills when dealing with more than one substantial subject or task at a time. Unfortunately, with this particular upcoming Friday being riddled with more than just this deadline, I will be scrambling to procure a pull request for this in the next day or two. 

At the very least, I'm in slight familiar ground with CSS and armed with a broad but tested strategy of element/CSS property inspection using web browser development tools to pinpoint where the snippet of code I'm looking for is located.

This epic cliffhanger will conclude with a blog post at the end of the week.

Saturday, October 4, 2014

Initial Bug Scavenging

Last week I've managed to scour the Appmaker repository for issues that might be my pace to start off with. I ended up finding Issue #2251 which in accordance with the recommendations of professor David Humphrey and seasoned mozillian Scott Downe, this seems to be the perfect scope for the deadline and for dipping my feet in the Appmaker environment for the first time. The size and priority of the bug itself are at a state where it is extremely unlikely to block anyone from continuing their current work on the codebase which also provides me with more time to acquaint myself with the logical hurdles and conventions that naturally baggage themselves with a new project I'm about to dive into. Perhaps if done early, I might be able to find another one before the 0.2 deadline.