2017 Tech at SUMO Heavy: Performance, Security & Data
This will be our seventh year as a team helping clients grow and maintain their businesses. We have learned a lot along the way — sometimes getting it right, and like other humans sometimes getting it wrong. One lesson that we always try to teach our clients is that saving time and money is just as important as making money. As a business owner, you may not be well positioned to focus on some forms of growth due to spending available funds elsewhere, but you are always positioned to save time and money which can lead to other forms of growth.
Keeping this in mind, we looked back on our years of experience both inside and outside of SUMO, and have come up with three areas that are often neglected. First, let’s get into the multi-faceted reasons these areas are neglected:
- Agencies often have thin margins due to over promising and working hourly, having to fight for each hour they poorly judged a timeframe for
- Many engineers and developers are solving engineering and development problems, but not business problems.
- Senior level management at an agency is often focused on selling, and not on their current client base.
- Many agencies specialize in one platform, such as Magento or Shopify. This not only leads them to wear blinders organizationally, but also stunts their growth as an organization and as a team by ignoring everything else that’s out there. This inherently puts those platforms’ organizations in a controlling position, causing overwhelming stress to tear through those agencies when something doesn’t go their way. These companies may be able to solve a “Magento” problem, but if you ask them to solve an inefficiency in a business problem, you could be out of luck.
This outlines what we don’t want to, and will not become. We have not, and will never become yet another agency controlled by the organizations and platforms available. Our job is to help our clients solve real-world business problems, and enable them to be in control of their business. Business owners need to take back their businesses, and not leave it solely in the hands of the organizations of their ERP or eCommerce platform.
This leaves us with those three areas of focus for now, and hopefully forever: performance, security, and data.
So, what does that mean for businesses?
These are the areas that we find consistently hold back our clients from either over-spending time and money, or from growing (and unfortunately, sometimes both). We like to boil ideas down into these topics because they do not apply to any one type of business. Whether you’re an online Saas product, a platform extension, or offline business, these three areas should be your focal points.
If you run an online shop, performance might be something as simple as page load time, or as complex as the time it takes for a customer service rep to look up a customer and access their information. In another scenario, an offline business such as a body shop is not performant if it takes an hour to rotate a set of tires when it should be done in 15 minutes. Our job is to help our clients make their overall business processes more efficient, and finding the point of diminishing returns along the way — if you’re already handling 10k concurrent users at 30ms, should you spend five hours to make it 29.5ms?). Here’s a simple scenario I’ve always found fascinating:
In 2014, we were working with a business that had a customer service problem. Their average call time was steadily increasing, killing the number of customers that could be addressed every day over the phone. Digging into it, we found that on average it was taking around 40 seconds to look up a customer record. Looking at it, their number of customers had grown to over 5 million and the lookups were slow because they were live, against a complex part of the database. Under certain conditions, we know that a LIKE can ignore indexes, which can be a performance nightmare. We were able to apply changes to the platform which resulted in lookups of 0.014 seconds. This was a two-fold performance problem: the customer service reps were not able to fulfill their daily goals, and the platform was not able to efficiently search just 5 million records (which let’s face it — in today’s world that’s not a very large dataset).
On a side note, this is one reason we work on a retainer model. We’re not going to focus on that here, but just think about this: Fixing this issue took about an hour. If someone were charging $500/hr, a solution like this could save a business millions of dollars every year, and would certainly help customer retention, lasting for years. This isn’t to say the client should be gouged, but it does say that knowing how to take quick action on an issue like this proves that your hour of time is worth more than $500. Knowing the value of experience is important.
We are lucky to live in a world that is becoming increasingly safer in many ways. Unfortunately in other ways, it is getting far too easy to be a bad person. Protecting yourself and your users is vital. We are committed to helping the community and our clients solve common security problems. Sometimes this results in OWASP or PCI/DSS education, and others it requires a deep understanding of the business’ technical architecture. The core of many platforms are severely lacking regarding security, and we want to help fill that gap.
Regarding one of the platforms we work with, there was a terrible vulnerability discovered last year, which resulted in credit card skimmers being installed on thousands of websites. We are developing an open source extension for that platform that will scan changes. Due to an architectural oversight, the platform primarily uses events in the code, when in some cases database triggers should be used. This extension is a simple example of how engineers and developers being more informed about business could make better decisions, and hopefully, this will inspire others to look at their platforms as well.
It’s 2017, and most companies still don’t understand how valuable their data are. Every year, warehousing and analyzing data become less expensive than the last, and it is still in many cases one of the last things people consider looking into when creating budgets. There’s also a large gap in understanding data warehouses versus real-time analytics. We are focusing heavily on creating formal taxonomy within our clients’ organizations, and then discussing the options that are a good fit for them.
This can be a difficult sell. As a consultant, you don’t always have access to a sample dataset of billions of records or at petabyte scale. That said, most businesses have missed out on mining many petabytes of their data. It’s hard to go into a room of non-technical executives and openly discuss just how great it will be to have this data.
We are developing a few solutions that integrate various platforms with common tools (Amazon EMR, Snowplow Analytics, Amazon Quicksight, and more) we can set up for someone and see results in a few hours. These solutions will hopefully help people understand how analysis can be done easier with the right tools.
Open Source Initiative
It has been a dream of mine to ‘open source’ as much as we can. Are we a little late to the game? Maybe, but what they don’t tell you is that open source is hard. If you’ve ever maintained an open source project, you’ll understand that not everyone is WordPress, or Ruby on Rails, or Laravel with hundreds or thousands of people committing. Smaller projects are mostly people complaining and never committing anything to your project. It takes personal time, a lot of effort, and passion. We are finally positioned to start doing this, and that makes me happy.
We’ve dealt with many proprietary systems over the years and will continue to do so, but due to larger organizations adopting more open source projects, I am proud to announce that everything we develop that we will use across clients will be open. We’re building a new website that will be home to all the projects and our engineering blog, and all projects will be hosted on Github. More details will be coming later this month.