TechTalk with Shadman Rahman from Keystone Education Group

Welcome to another tech talk where one of our clients shares their experience working with the QAwerk team. Expect to find out what it’s like to build and test an education marketplace for millions of users worldwide. You’ll learn about the daily challenges, testing areas that require the most attention, and the results you can achieve with a dedicated QA partner working alongside your team.

We’ll also touch upon related topics, such as what it’s like to work at an international organization, the latest tech trends, and what work-life balance looks like for a senior product manager. This tech talk is jam-packed with valuable insights, so be sure to read (or watch, if you prefer) until the end!

Meet Shadman!

We invited Shadman Rahman, Senior Product Manager at Keystone Education Group, to talk about what it takes to oversee the quality of a global education platform. He also shares what life is like at Keystone and why having a partner like QAwerk has been a winning strategy for them. Let’s dive right in!

—

Hi, my name is Shadman, I’m a Senior Product Manager at Keystone Education Group, and one of my biggest responsibilities and roles is to look after the search and discovery experience for our international sites.

—

What drew me to Keystone actually was the mission. The mission of connecting with the students and making their education dream come true because I myself have been an international student here in Sweden. So, it only made sense for me to sort of come here and make other people’s dreams come true.

—

I really wanted to become a part of this particular company because of the international culture, to be really honest. In our office, you will see that we have the French team, we have the German team, we have the US team, we have a domestic Swedish team, we also have teams in Denmark, Finland, the Netherlands as well. So it’s a very multicultural environment and me being international, it was really helpful for me to sort of blend in to the whole environment. It literally feels like home away from home.

—

Being extremely student-centric. We are a very user-focused company. We are the marketplace where some of our customers are universities and educational institutions, and then, on the other hand, we also have the students as well. Particularly for me, I’m driven by the idea of being in the middle, but I’m actually connecting the dots and helping them find the right match, which is why the search and discovery experience in general, and that is what motivates me to be engaged day to day.

About Keystone and Its Mission

—

Keystone Education Group is an education marketplace, to put it simply. What we do is we help schools and educational institutions find the best possible students all around the world.

The core problem that Keystone Education Group is trying to solve for students is to find their best possible match in terms of their education, specifically, higher education. The moment we graduate from our school, the biggest responsibility for us is to get the best possible education that we can dream of. And our goal is to help you find your next best dream location for you to study, be it a school, be it a new country, be it anywhere and anything that you like. Our goal is to make your dream come true.

—

So, this is where our development team and our product and technology team basically collaborate. We meet here once or twice every week and we are sort of distributed. So, we have offices across other countries as well in Norway, in the UK, and definitely we also have a lot of other, let’s say, individuals located in Poland and in other geographies as well.

Inside Keystone’s office in Stockholm, Sweden

But whoever is in Stockholm, we try to come here and meet once a week at least to sort of mingle around and collaborate ideas in real time.

—

A normal day for me is definitely filled with meetings, a lot of collaborations, a lot of cross-functional communication, and storytelling. And what I try to do is try every single day, we try to take some bit of a time where we actually spend some time talking to our real customers. And also we take some bit of time actually using our own product so that we can fall in love with the product that we already have built for our customers.

When it comes to overseeing multiple web platforms, one of the key aspects that we try to keep in mind is the different markets, the different localities and how they behave. German, French, Spanish and Scandinavians or Americans, we are, yes, we are similar, but also we have our own identities as well.

So, when it comes to looking at multiple different projects, it has to do a lot with understanding the market, understanding the context. And that is what we try to do on a day-to-day basis, honestly speaking.

—

The company culture in Keystone, first of all, is truly multicultural, truly international. So that is one of the biggest perks, I would say, being able to work at Keystone. Apart from that, it’s a very open and flat, let’s say, organization, which means that we are having lunch together across multiple teams.

Keystone’s teammates connecting during a coffee break

We are mingling, socializing. So at one point, basically, it feels like a global citizen environment, honestly speaking. So this is something that I really, really enjoy and really, really love about Keystone.

Keystone’s Approach to Quality Assurance

—

Before QAwerk, our quality assurance was mostly driven internally, meaning we ourselves, our product managers, our developers, our designers, and also our sales and customer service team members were basically doing the manual testing themselves. And very soon, we started realizing the fact that this is not a very scalable solution, as we can imagine. And that’s where QAwerk came into being.

The reason it was not a scalable solution for us to do the manual work ourselves was basically because we only cover certain common use cases when it comes to using the product, handling the different technicalities of it.

The limitations to that approach before QAwerk was, first of all, a lot of manual work. Second of all, we only covered common, let’s say, product journeys and use cases. But we all know that there are hidden secrets all across the different corners of the product itself. We needed someone to help us unearth those secrets, unearth those areas. Also, we needed someone to actually challenge us to be better.

—

One of the key challenges that we were facing back then was all the different languages that we were offering. We offer our websites in close to 40-plus different languages, 46 to be more precise. So as you can clearly imagine, for every single test, we needed 46 different individuals or different nationalities for us to be able to go and test every single corner of it. That definitely and naturally was not a very scalable situation for us. So localization, I would say, was one of our biggest key challenges. Hence, QAwerk.

—

We are all familiar with those Google Translate memes, right? Where we see people making fun that Google translates in a very wrong way. That is one of the biggest challenges with machine automation or machine translation, which lacks a lot of context, which lacks a lot of different variabilities. And also, there are issues like a certain word, probably in Chinese, a character, is this much. But in, let’s say, in Finnish, it’s going to be this much. In Hungarian, it’s going to be this much. So these are all, let’s say, little nuances that we had to sort of constantly look for.

And by the introduction of QAwerk, we were able to quickly spot those areas. We were, as we started repeatedly facing those areas, we started repeatedly considering them when it comes to new feature development, new product development. And as a result of that, we saw a tremendous growth in our organic traffic.

Not only on the student side, but I would rather say that we have also seen a tremendous impact on the university partnership side for us as well. How, you may ask? The answer is basically that we were able to rapidly offer them a multicultural or a multilingual experience that was very hard to find in the overall market.

In fact, I would probably say we were the only ones that were offering them, actually, let’s say, offering 46 different versions of their own content and their own programs. So, that definitely helped us improve a lot more trust and gain a lot more credibility being a trusted partner for all the educational institutions as well.

—

When it comes to feedback, the number says it all, honestly speaking. As we started seeing fewer bugs, as we started seeing smoother processes, we could really start having fun conversations with our university partners. And we also saw good, healthy, organic traffic on our sites as well. And that definitely made all our crossfunctional team members super happy as well. So in a way, it has been life-changing.

—

When it comes to QAwerk, catching something just in time before the release, one immediate thing that pops up into my head is one of the most recent finds. We were about to change, make a huge change in our URL structures. And right before the release, we wanted QAwerk to actually come into our staging environment and test the overall experience and the issues that they had been able to identify. Honestly speaking, we wouldn’t have been able to do so. And that was, I would say, it saved my job. So thank you QAwerk.

Collaboration with QAwerk

—

We decided to bring in an external testing agency rather than building our own in-house, let’s say, capabilities specifically because of getting rid of the biases, honestly speaking. Once you are in the environment, we are seeing…when we work, we get so deep into the nitty gritties and the functionalities of a little feature.

What happens is sometimes you overlook the overall, let’s say, picture. So it’s like looking at the tree versus looking at the forest. So we thought that an external partner in this scenario will be able to identify the right, areas and help us pick out the areas that we need to improve from a more neutral perspective.

—

The nuance of the multicultural localization aspects. Being someone who is able to offer QA services in several different languages, someone who has a thorough experience in understanding different markets and how content may differ. Someone who understands the marketplace model in general. The business model as a whole was extremely important for us.

—

We chose QAwerk over other vendors, first of all, as the Gen Z say, because they passed the vibe check. So I would rather say in this scenario that we had a long list of demands. We wanted someone capable of understanding different localization. We wanted someone who had thorough experience in end-to-end testing. We wanted someone being able to understand the marketplace business model as well. And QAwerk in pretty much all aspects has been able to check all the boxes with flying colors.

—

When it comes to being a little bit hesitant, working with an external partner, yes, of course, it begins with a lot of skepticism, to be really honest. And we had our own set of assumptions as well. But I would rather say that as we started sort of working with QAwerk more and more, right now we don’t even think about them as being an external partner. They have become part of our family. And, in fact, I would say we look towards them for counsel. We look at it as they are the third eye that helps us be better.

—

Our views on outsourcing have significantly, I would say, not just changed. The right word would be enhanced, rather, because now you can call us bigger advocates. Because one of the hypotheses that we had was maybe someone from the outside will be able to give us a fresh new perspective, fresh energy to actually make us better. And I think with QAwerk, they passed all the checks with flying colors from that perspective. And that now makes us feel super excited to be able to sort of, let’s say, really work with external partners in this scenario.

—

I would say that previously the process was a little bit messy and a lot manual. But now it is part of the definition of done when it comes to shipping any particular product or developing new features. So the way we work is when we are thinking of an idea, when we are brainstorming about it, our sort of thought of QA starts from that point till the point where we hit the go-live button. So they are part of us throughout the process, basically.

—

One of the biggest changes that has happened at Keystone when it comes to QA is the whole process. Previously we did not even have a process. Right now we have a proper structure in place where QA is along us throughout the entire journey.

—

QAwerk’s communication and reporting have always been extremely transparent, and I would say that their version of communication is overcommunication rather than undercommunication, which is a great thing in at least our context.

So what has significantly helped us is the monthly reports, where we get an overview of how many hours and which are the areas we are looking into. Also, we have an impromptu communication channel with them where all of us, people like the product managers and the developers, are all connected through Slack, and we are constantly communicating with the QA personnel on the other end in real time —and that has significantly helped us ship our products way, way faster.

—

When it comes to testing strategies and methods from QAwerk, one of the things that has significantly helped us is the thorough end-to-end testing because on our end, we do the functional testing, do the integration testing, but the end-to-end testing is something where we really need a fresh pair of eyes, and that’s exactly where QAwerk shines.

—

One of the most valuable impacts from a technical perspective has been on our Core Web Vitals, which is basically Google’s way of measuring how good of a website you are when it comes to search engine optimization. I would say that since we offer our products in so many different languages, being able to quickly do all the testing in different languages has helped us improve our overall Core Web Vitals.

—

The best part about working with QAwerk and the QA team in general, honestly, is the people. They are one of the most talented, amazing young individuals that I have come across. And what they do day to day, day in and day out, only makes us better. And for that, I am definitely ever so grateful to them.

—

My recommendation about QAwerk would be to anyone and everyone out there who is looking to make themselves better by challenging themselves. Working with them is definitely not going to be easy. You know why? Because they will challenge you. They will challenge you to be better. And trust me, that is only going to be good for you.

Technical Challenges and Future Vision

—

When it comes to localization and all the complexities with 40 plus languages, one of the biggest challenges is that we need them. Why? Because we are a global platform and our goal is to connect probably the most remote part of the world to the best possible match they have got.

Maybe someone from Indonesia is a perfect fit for a particular hospitality program offered in Mexico, for example. So, for that Indonesian student to feel absolutely comfortable to browse our sites in their own mother tongue creates a sense of trust for them versus a third language. Neither Spanish from Mexico nor Bahasa Melayu in Indonesia but rather English definitely creates a sense of hesitation for them. So, from a strategic perspective, being able to offer different languages to make students feel at home is extremely crucial for us.

—

QAwerk has helped us refine one of the most critical journeys for a student, which is the search and experience. For someone who is sitting in front of the computer and going on Google and deciding, “Hmm, I would like to get a Bachelor’s degree. Let me see what my options are.” And when they are able to see the search results in Google for something in their own language, that naturally draws them towards that particular website. That has been our strategy, and QAwerk has been able to help us refine that journey end-to-end.

—

One of the most important things that we look into is the uptake of organic traffic. Because organic traffic is something that is naturally coming to our platforms. And they are the biggest drivers. And I’m super happy about it myself as well.

And apart from that, one of the key aspects that we look for is the Core Web Vitals because, as a technical product, the vitals need to be stable and steady. What we also look for is a seamless experience across different localizations and across different languages that we offer. And when it comes to the ultimate North Star metrics, we are a traffic-and-leads business. So, the number of conversions is one of our key metrics when it comes to looking at the success or the satisfaction of our users.

—

One of the key pitfalls is the localization aspect because it is not just a Google Translate or an automated machine translation of what a particular word is—every single language has its own nuances.

In Thailand or in the Thai language, there is a particular way of reading between the lines; it is extremely important. Versus, if we look at the Dutch culture, they are known for being extremely direct. You can clearly see the two different contrasts over here. Now imagine that in a scaled level of 46 different dimensions. So, this is one of the key challenges and a key pitfall when it comes to scaling these businesses.

—

When it comes to solutions to such pitfalls, one of the key areas and one of the key aspects for us to look through is understanding the cultural context, understanding the nuances, what actually works, how people perceive a particular piece of content when they are reading it. So, it all boils down to the nuances of the languages. Unless you are a language expert, mastering localization is extremely difficult.

—

Here at Keystone, pretty much starting from product discovery to delivery, everything is done in-house. One of the key aspects that we have outsourced in this scenario is the QA, and one of the key reasons for doing so is for us to gain a third-eye perspective. As we are a global company, as we offer our services across the globe, we thought it to be extremely important for us to be able to look at ourselves from different eyes, and that’s where QAwerk comes into play.

—

At Keystone, our plan for the future is to take localization at scale. We want to be able to offer a destination, an education destination for every single country, every single language out there. In order for us to be able to do so, having a third-eye perspective and having a QA that helps us refine end-to-end search and discovery journeys, not just search and discovery, but also all the other journeys of an entire marketplace is extremely crucial.

Industry and Leadership Insights

—

One of the biggest tech trends as of right now that excites all of us the most is definitely the emergence of AI. But for me, I am more interested in understanding how through AI we can offer localization at scale. So that is something that I’m truly looking forward to.

—

The best way to communicate something within an organization, I would say, is passion, enthusiasm, and full transparency. That is what makes every single conversation trustworthy, every single conversation meaningful and heartfelt.

—

Outside work, or whenever I’m not at the office, I’m a full time dad. I have twins, a boy and a girl. They keep me fairly busy. But apart from that, one of the key things that I’m most excited about is my second persona as a musician. I do love playing a lot of different instruments. I mingle a lot, or let’s say, tweak a lot with different kinds of music. So, that also keeps me sharp, keeps me busy, and also keeps me entertained.

—

If you really want to understand what goes on in your user’s mind, I would highly recommend reading Continuous Discovery Habits by Teresa Torres. It’s a book that changed my overall lens of how I look at a product and also how I look at users in general.

It helped me gain a new perspective on opportunities that are outside of the pain points. As I was learning the art of product management, I would say that we are very obsessed with problem statements. But what Teresa Torres has done is change that viewpoint into user opportunities that we can actually make an impact on in general. And that is something that I believe that can help all of us be better.

—

Something that is very true, but no one agrees with me, is that I am some sort of an introvert. Whenever you ask anybody, they will say, “Shadman, introvert? Impossible!” But trust me, there are moments in my day where I feel I need my quiet space, I need my area to recharge myself as well. I am not always the social butterfly as people see me to be.

—

There have been so many difficult decisions in my career, and so many turning points in my career. But I think one of the hardest, most difficult decisions that I had to make was when I decided to switch from being a UX designer to a product manager. I transitioned into this role with a lot of optimism, with an equal balance of pessimism as well, because I knew as a designer, when I was a designer, I wasn’t very fond of the product managers that I dealt with. So, taking up a role where I didn’t even, let’s say, have good relationships with earlier was a huge risk that I took, let’s say, six to seven years ago.

—

So, my life motto in general is very, very, very simple. I live, I live to serve my team, I live to serve my kids, I live to serve my family. That’s what gives me pleasure and that’s what gets me going.

Closing Thoughts

Have you ever faced the same challenges Shadman described when trying to ensure the quality of your marketplace or web application? Perhaps you’ve experienced a lack of a proper QA process, stretched your internal team too thin with laborious QA tasks, or noticed bugs slipping through the cracks due to tunnel vision. What if your releases didn’t have to be chaotic, stressful, and unpredictable?

Contact us today to share your QA challenges and schedule a complimentary consultation. You’ll get a clear understanding of what you can change right now. We’d be happy to have you among our satisfied clients, such as Keystone Education Group, whose investment in quality pays off one student application at a time.