Continuing with the delve into Climate Change, looked into understanding how people feel, positive/neutral/negative, about the subject on Twitter. Using TextBlob, with Tweepy, Pandas, and Plotly Dash…I built initial revision of a data visualization dashboard, Climate Mood, that analyzes tweets over a sample of the last 2,000 collected hourly. The idea is to provide a simple, sentiment-focused look across multiple analyses. I’ve particularly enjoyed building and re-applying some of the methodologies and statistical analyses I’ve learned recently.
As mentioned, the dashboard pulls from Twitter over an hourly basis and deposits into a PostgreSQL database. The top chart provides a linear regression trend line (unevenly for now) across tweets collected that can be toggled. Also, tweets can be seen on the top-right tweet-box on hover. Other simple sentiment analyses include average verbosity of tweet length, popularity of tweets, and average sentiment-based amplitude of tweets. Feel free to check it out!
Looking forward to applying more advanced statistical analyses including a live Monte Carlo Simulation calculation across these auto-updated samples…among others. If you have any feedback or suggestions, please submit an issue on the GitHub repo or contact me.
Been thinking about climate a bunch lately and started looking into trying out the latest data tools and trends out there on the Internets. I’ve particularly enjoyed R and Pandas/Python, especially via Plotly Dash, and re-applying some of the methodologies and statistical analyses I’ve learned while on the job. The latest curiosity is in global temperature change since 1910 and various other societal factors. Lo and behold, the first iteration and not-so-polished, Climate Trends.
The dashboard has a drop-down menu of various analyses around global temperature change and linear fits across birth rates, fertility rates, death rates, life expectancy at birth, and rural population growth rate as well as a nominal years passed chart. Feel free to dabble!
If you have any feedback or suggestions, please submit an issue on the GitHub repo, respond on Dribbble, or contact me.
Near-death experiences provide a heavy emotional weight…even 16 years later and, generally, the rest of a person’s life.
Still remember everything, as much as I can, from that night in Irvine. My most important anniversary thus far in life. Here goes:
It started with a pecan-filled brownie that wasn’t properly listed with its ingredients. After eating it at Mesa Commons, a half hour later I started experiencing all sorts of issues: swollen throat, diarrhea, vomiting, blood-red eyes, swollen forehead (that was red too through my brown skin), a distinct lack of oxygen, inability to walk without support, and an inability to breath or talk. I needed to type out, letter barely after letter, to take me to the hospital by my dorm-mates. I lost consciousness before I got out of the door. My dormmates told me that I fell apart in their arms at the steps leading to the parking lot and fell down a flight unconscious. They called the paramedics and I got a number of Epipen shots to kick my body into gear…ended up 30 minutes to an hour later at the ICU at Irvine Hospital. The docs took awhile on me and tried out some experimental drug to open up my lungs and throat…a last ditch effort before calling me dead, actually.
Thankfully, my lungs and throat opened up. 3/4’s of a day later, I woke up from a coma that the doctors weren’t sure if I’d ever wake up from with a family crying (they drove all the way down overnight not knowing if I was going to be alive or dead by the time they reached there). I was intubated and lying there motionless not able to move a muscle as my body was wrecked and lifeless.
It took another day to move me out of ICU and into a proper bed without intubation or…anything else needed to support me. I couldn’t walk for a day, but I told my parents, well, I had homework I needed to get done by Monday for my Physics class (one of the first things I said after getting out of the coma). I stayed 3 days in the hospital and my parents, incredibly begrudgingly decided to keep me in Irvine (400 miles away and only 4 months being alone for the first time in my life). I went to my physics class in a crutch unable to walk properly, but I was there at 7:30am in the morning with my homework.
From there, I ended up on academic contract a quarter later. I survived that as well taking the toughest grading professors in the Computer Engineering program for 3 straight quarters. Ended up graduating with a pretty decent GPA after it and a lot of discipline built up over the years and through that.
Heading to India for a vacation and now have a chance to try out a lot of different foods, hopefully, and check out places I haven’t been able to previously. Something I’ve missed out on in my life as cultural events and family parties were kind of a death wish due to the various nuts, beans, and seeds that were prevalent. Thanks to my awesome reduction in allergies, have a chance at reconnecting with my parent’s roots as well and seeing things as what they went through. Even put together a spreadsheet/rubric on things to do and eat there (as well as various transportation options that are pertinent).
Have been to India in the past, but it’s also been limited. This time, there’s happiness and excitement to something I’ve always wanted to partake in. Very excited about the trip and looking forward to seeing family as well as trying out a lot of “new” without any deathly allergies!
I was talking to family and friends about various investments made (and some missed out on) over the years. Have gotten decent at picking winners (using a Value-Investment framework inspired by this book that mixes intuition, selection, and analysis). Here’s the base-level rubric for what I look into before I investment into any company. Some analyses go further. Generally, 9 months to 1 year is taken before any action…essentially driving the associations of the entity into muscle memory and relying on intuition, alongside any other company/product/market, to make the call.
Read on 3-6 months of articles on stock and target company
Read on 3-6 months of analyst coverage on stock
Find Market Cap potential of target company
Find Market Leader
Identify Competitor Set
Identify Lead Competitor(s)
Find Growth Darkhorse(s)
Identify any market regulation
Determine CEO capabilities and habits
Determine CEO’s style of communications (past 2 years)
Identify Product Roadmap
Read last 5 product announcements
Linkedin Lookup on Corporate/Employee Composition (specifically product/marketing/engineering)
Read past 10 10-Q, 2 10-K
Identify, if possible, Sales & Marketing + R&D spend
Identify, if possible, Growth of S & M + R & D spend
Would love to hear your thoughts! Feel free to DM via LinkedIn.
I’ve written about Tesla’s growing community of advocates back in 2013…well, it’s now becoming real. Though is it happening because of an explicit push or something happening naturally due to increased sales?
My hypothesis is that I think Elon is being Elon and going on a major PR push in 2018.
Been tracking since February and it keeps on confirming along the way. With the Starman launch (February), Grimes (May), and “Pedo” (sigh) (July), I think there’s something going on in terms of media coverage on various issues in the US/Worldwide, and Elon Musk is either trying to take advantage of it or doing his own thing. Feel free to add the search term “pedo” to this Google Trends chart…I won’t.
Maybe this is Elon “taking control of the narrative”?
I haven’t done a multivariate analysis with Tesla’s share price, but it might be worthwhile now. Once the shorts/media started their campaign last year, it became clear that Tesla’s share price was going to be affected in a hedge-fund kind of way. I think this company should be valued, at least, at 2-2.5x higher than the current by end of 2018, if the market is accurate and corrects itself.
SJ Charts is available now fully open-sourced here! Plot.ly has been fun and the documentation, as well as the community forums, have been great to work with. Wish there was more in-grained analytics support via Google Analytics and Mixpanel, but that seems to be a work-in-progress. Here’s what it looks like now:
The charts now have time-based charts for Housing Prices, Unemployment rate, and Jobs by Sector for the City of San Jose. The idea behind this app is to show various analyses via a top-level tabbed navigation for the City of San Jose residents. Basically, to build out useful analyses for the open data available from the City of San Jose.
So, I took a few weeks to get on-ramped with Pandas 0.23.0 and Python and am still learning the ropes. Though, I wanted to share some output. Namely, using the San Jose Open Data Portal, there were some good insights to be had from looking at economic data pushed out by the city for 2006-2016.
What did I do?
Looked at unemployment rates across various housing prices (condos/townhomes & single-family homes) using ordinary-least-squares regression.
Looked at, over time, labor force and housing price changes.
SJ Economic Changes, 2006-2016
Condo/Townhome prices vs. SJ Metro Unemployment
Single Family Homes prices vs. SJ Unemployment
What did I find? Namely, housing prices are increasing rapidly compared to labor change (i.e. job growth) in San Jose over the course of 2006-2016. Also, there’s a strong correlative relationship between something that we all take for granted and as obvious: increased housing prices in San Jose runs with lowered unemployment rates.
If you’d like to look at the data analysis done, here’s the GitHub repo. If you have any questions/comments/suggestions on more analysis, feel free to contact me via Twitter, GitHub, or LinkedIn.
I’ve been working through a few updates on a prioritization framework for use in a feature and sprint planning session. The very nature of the sessions are to align on what a team is going to be working on with ensuring it’s appropriated towards the highest priority items. After a few recent chats with other PM’s, there was a needed sense of balance between business needs, product fit, and cost.
Here’s the latest update that aims to provide that sense of balance:
ROI – Return on investment of fitting product theme fit, churn
potential, and roadmap fit relayed to a weighted average against t-shirt
Cost – These are the t-shirt sizes that engineering generally comes up with to assess viability of a themed story or issue.
Legend – It’s an updated map or legend that provides PM’s and Engineers the ability to map back specific priorities to the ROI number.
I’ve been on a brief hiatus for a couple of months
and have been thinking about the next endeavor. After some personal time off (even now), there was one idea that’d come to mind. The hard part about it is getting feedback from a product perspective. How do you do that in a way that garners enough attention, brings in interested individuals, and siphons the idea into something usable?
So, I’m writing this as a thought experiment to see what feedback I could garner for an app idea (and prototype). Namely, with public service infrastructure, the hardest part about getting engaged with an institution is how do you reach them and about what? There’s some movements in the Civic Tech space with Open 311, but how do you talk about moments of epiphany about positive comments or questions?
3M residents in silicon valley; 1M in San Jose
50 requests incoming through mobile apps; 1,000’s coming through phone
Inefficiency issues in getting the right feedback
Inefficiency in the right departments receiving the feedback
Customer Service Directors — Government employees who run day-to-day operations for customer service departments within departments in a municipality (both city and county). They need to quickly triage and propagate work incoming requests in an efficient and effective manner.
Community Relations Directors — Government employees who directly interface with the public in regards to social media and email. They need to ensure the department and municipalities are directly interfacing with the needs of their constituents and are kept happy with the service provided. They want to provide greater engagement and quality interactions with citizens and department staff on feedback and requests.
Residents — Citizens and individuals within specific geographic regions that depend on public infrastructure to live and perform tasks day-to-day. They need to have infrastructure to run efficiently and effectively. They want not have to interface with their public works departments, but understand that it takes feedback in order to populate how well the infrastructure is running.
Community Relations Directors: Customer service responses don’t go to the right departments.
Customer Service Directors: Customer service requests don’t have enough information to be serviceable.
Residents: It’s difficult to send customer service requests and feedback to public agencies.
Provide a mobile app experience that adds a description, location, and photo. Using a matching algorithm, provide recommendations on departments to send the request to and sends a customer service request to individual public service departments. Each request requires a verified e-mail address and, potentially, a physical address.
First, target individual geographic regions, such as Silicon Valley, to improve on ways to match filters to recommended departments. Stick to regions with a distribution strategy of tech-savvy, civic oriented residents and focus on garnering normative samples of early adopters. Build up regions supported based off of data-informed recommendations from feedback from the user base. Then, expand using volunteer groups and customer service departments as advocates of a free mobile app
across iOS and Android.