Category Archives: Software QA Testing

Leveraging Kubernetes for End-to-End Quality Assurance

Quality Assurance, Software QA TestingAjit Singh

Kubernetes is an all-in-one container orchestration tool that helps in the management, deployment, monitoring, and scaling of systems for containerized applications which are all packaged in one open-source system. If your organization works with containerized applications using Docker, then you would already know that things become tedious, and it becomes difficult to manage them when they scale up.

……………………………………………………………………………………………………

Classic QA Process:

Here, the end-user only interacts with the front-end user interface in both monolithic and containerized applications. Similarly, the traditional QA process can be used for containerized applications but should be seen as a subset of the QA process in the Kubernetes world. UI or API testing using tools like selenium or Rest- assured can be used for automation testing for applications running on Kubernetes, but this is a costly process that requires high investment in terms of time, money & resources. To overcome this, it requires us to look outside the box to find better solutions that can be paired up with these and provide better coverage right from the beginning. This would result in the scope of the QA process getting increased when applied to Kubernetes applications.

Enhanced QA Process (hybrid):

Kubernetes applications are generally deployed using continuous integration and continuous delivery. To assist the developers in the release cycle, the traditional approach of creating end-to-end tests is not adequate and now requires QA to work very closely with developers on the tests so that it can be written quickly and executed swiftly.

Hybrid QA Process

……………………………………………………………………………………………………

The following are the new technologies that help in accomplish the full potential of the QA process in Kubernetes world.

  1. AB Testing
  2. Chaos Engineering
  3. Consumer-Driven Contract Testing
  4. Helm Tests
AB Testing:

The complexity of applications has increased vastly, and business requires constant UI updates and feedback based on the frontend changes. The business stakeholders hire marketing and research teams to attract more customers and convert more leads from different marketing campaigns.

Here Kubernetes gives us the power to leverage continuous feedback by using the customers as testers and present real-time statistics for businesses to make the decisions in real-time

AB Testing

Chaos Engineering:

Chaos engineering is used to test the resilience and weakness of a system in production by creating expected chaos in the system to verify that none of the activities has affected the performance and stability of the system. This builds confidence that the system can handle unexpected situations and any failures that exist can be addressed before the customers face the brunt.  Since the get known to the QA team in advance, it gives them the opportunity to fix those unexpected outages before they actually affect large groups of users. Some examples can include scenarios like increasing load on a Microservice, introducing DNS failures, Mimicking hard drive failure etc.

This can be achieved by using Free and open-source tools available depending upon the cloud provider on which the system is hosted. For example, one of the better-known tools is Chaos Monkey used by Netflix for cloud-hosted Kubernetes clusters.

Chaos Engineering

Consumer-Driven Contract Testing:

Often times while working with microservices, creating full end-to-end integration tests are expensive and require high execution time. Even if you hit failures, it is hard to figure out which service is causing the failure if the complexity of the system is high. We can create Consumer-Driven Contract Testing using Pact which is available in almost all languages. It makes it easy to write the integration tests and helps us to test consumer-provider rules using JSON files.

So, what is a pact? Well, PACT is a binding contract between the consumer and the provider, which has to be always followed. In the world of microservices, the pact is based on the API call (consumer) and response (provider) and can be done using either a single isolated microservice (while mocking the other microservices) or on a real system with all microservices running.

Why use PACT when we can write end to end integration tests?  Well, it helps us create tests before we move to write expensive integration tests and can provide us general sanity of all the microservices running in our system.

Consumer Driven Contract Testing

Helm Tests:

Most of the organizations use Helm charts to deploy Kubernetes resources on Kubernetes clusters. These charts are really powerful and often contain tests written in the chart which can be used to test if all the resources got deployed successfully to our Kubernetes cluster.

Since Helm tests use a scripting language to perform basic sanity checks on the deployed resource, we should leverage the tests provided by popular charts over the internet. In case, you are making in house helm charts, then you might want to skip this for some better integration test since tests written in scripting language can be difficult to write and maintain.

……………………………………………………………………………………………………

With the fast evolution of the application architecture and introduction of DevOps culture, the traditional QA process is not able to provide the same value for money and is often seen as one of the bottlenecks when it comes to the release cycle. Hence, QA must design and plan their framework to provide a blend of fast test results and improved end-to-end integration tests. The role of QA is now changing from Quality Assurance to Quality Assistance. Since the QA process has to be optimized as per the AUT, this article hopefully has helped you in identifying the process of robust end to end QA framework for applications running on the Kubernetes cluster.

……………………………………………………………………………………………………

The views and opinions expressed in this article are those of the author. To know more about our company, please click on Mindfire Solutions.  For over 20+ years now, we have been the preferred Software Development Partner of over 1000+ Small and Medium-sized enterprises across the globe.

 

Spread the love
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
RPA Image

Is Robotic Process Automation changing the Test Automation Game?

Automation Testing Tools, IT Outsourcing, Software QA Testing

RPA has taken the IT world by storm. I won’t say that it is the newest thing in business because it has been there around for about 10 years now. But now is the time when it is spreading like wildfire. More and more companies want to adopt it.

To set the context, let’s look at how AI has broken all the barriers of our imagination. I bet that a few years ago when we used to see robots in movies, we would think that they are just part of our imagination or even if we considered them to become reality someday, we always thought that the day to be many years away. But what we never imagined was that technology will progress so much so soon. Today Artificial intelligence is making machines more and more human-like. They want machines to not only follow our instructions but to think, and possibly also to exhibit emotions. It is almost like humans want to assemble humans in labs. No wonder then that a robot named Sophia was very recently even given citizenship of Saudi Arabia.

Now going back to Robotic Process Automation. Going just by the words,  it seems like  Robots automating processes. The catch, however, is that that when we say Robot, we don’t mean an actual physical robot, but a virtual one –  the automation program.

……………………………………………………………………………………………………

What’s the big deal about RPA in Software Test Automation?

Many could argue that in traditional Automation also we were creating automation programs. So what is all the hype about?

The key distinctions according to me are :

  • It is script less: Coming from a Manual testing background it was always a challenge for me to always keep learning the latest scripting languages but now I don’t need to learn any programming language to automate a test scenario. You just need to be good with your logics and should be able to think out of the box scenarios.
  • Focus is back to product quality: I am not saying that in Traditional automation the product quality was getting hampered, But in my own experience I feel that when I was working on Automation via scripts, most of my time was getting consumed in writing code and at the end of the day, only very few scenarios were automated, and I was not able to cover as much scenario and I didn’t have the time to dive into variety of scenario. So I feel that out of two 1 thing was always getting compromised either the Coverage or the deadline. But with RPA tool this problem of mine has been resolved to a great extent.

……………………………………………………………………………………………………

How things work when Automating Test cases with RPA

There are many RPA tools available in the market like UI Path, Automation Anywhere, Blue Prism and many more. You can use any (FYI: I am not promoting any specific tool).

There are few basic criteria on which all the RPA tools are based on:

  • Already defined user actions: Most of the user actions that one can think of while automating a test case like the click of a button, mouse hover, opening browser, typing into a text box and many more, they are already defined and the user doesn’t have to code.
  • Built-in decision logics and looping statements: The best part about using RPA tool for creating scripts was, I didn’t have to worry about the syntax of my decision logic and adding loops, for example, do-while, for, etc. They are built-in and I just needed to use them.
  • Configuring of user events through Parameters: Every user action or event that we use can be easily configured using its parameters. Almost every property of a user event is made available so as it can be customized as much and as easily as possible.
  • Easy to create variables: RPA tools make it very easy to create a variable without stressing upon the syntax.
  • Error handling: If there is an error that occurs while configuring the user events or in the full flow, RPA tools have very good error handling mechanism. It helps the user to narrow down the area of error and also gives very clear and specific error messages making it very convenient for the user to correct the error.

……………………………………………………………………………………………………

What RPA has changed in my Automation Testing approach?
  • So as I said above, for me RPA has put the focus back to thinking about logic, covering some out of the box test scenarios, rather than spending hours just automating the basic functionality.
  • RPA has improved my testing coverage also, as I am able to cover number and different types of scenarios.
  • Thanks to RPA, I am able to automate the test scenarios faster as I don’t have to write every single line of code on my own.
  • Another advantage of using an RPA tool is, I face less number of errors while automating a test case. As in RPA tool, they have built-in codes for most of the logics and I just have to make sure that I use them in correct flow.
  • With RPA tool, I was able to manage my scripts much easily as most of the times it just making tweaking into the properties of the built-in user actions.
  • Also when I create my scripts using RPA tool, they are more easily readable and I am able to explain them to another person in a better way
  • Being from a manual testing background, it was always a big challenge for me to always keep on learning the latest scripting languages but now I don’t have to invest my time learning the coding languages.
  • I can invest my mind in more decision making tasks rather than some boring and repetitive tasks that can be done by machines also.

Now, when I extract the essence of all the benefits, the bigger picture that I get is RPA helps to increase Test Automation Coverage, reduces the time required, hence reducing in the process the cost of Testing and in turn increasing the Company’s profit.

……………………………………………………………………………………………………

Having said that, I did face some challenges while automating test scripts with RPA tool

1) I was not able to automate everything
Having worked with different scripting automation tools, I feel that there are some scenarios which I can’t achieve using RPA tools like I am not able to automate scenario which deals with complex Database entries, multiple formats of input or unstructured input data.

2) It executes at a slower pace
When I execute a script created via any RPA tool it executes at a UI speed but when I run a script created using scripting languages, it is much faster. So I feel that RPA is comparatively slower than the processes automated using traditional automation.

3) Not much available on Web, so need to explore yourself
Test Automation using RPA is relatively new. So there are many built-in user actions and functionalities of different tools that you have to explore on your own and not much help is available on the web. So that was also one of the challenges that I faced.

4) It can increase a company’s Test Automation cost
When I started automation using RPA, most of the RPA tools that I came across were paid and just 1 or 2 had free versions available and that too had not much-advanced features contrary to the open-source scripting tools available in the market. So I feel that this can also prove to be a disadvantage as it will increase the cost of Automation for a company.

But I feel that the pros somewhere outnumber the cons and the proof of this is that more and more companies are investing in Test Automation using RPA. So in my opinion, if you are the one who wants to adopt RPA in Test Automation, then the time to act is now.

……………………………………………………………………………………………………

The views and opinions expressed in this article are those of the author. To know more about our company, please click on Mindfire Solutions. 

Spread the love
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Google Puppeteer

Puppeteer : Is it time to ditch Selenium?

Automation Testing Tools, JavaScript, Software QA Testing, ,

Google has introduced a new library based on Chromium’s dev tool protocol. The library grants high-level API to interact with chromium and handle most tasks which you can perform using Chromium dev tools. The library is implemented in node.js and is really easy to get started once you install required packages using the node package manager.

……………………………………………………………………………………………………

Why Puppeteer?

Puppeteer flexes its muscle where selenium misses out. It is not just limited to interactions with the web application under test performing tasks like for e.g. interacting with dom, submitting forms, generating screenshots and pdf of web pages but also in intercepting network activity, throttling CPU and network, generating timeline trace and mocking API amongst some. The puppeteer can be easily be used to write end-to-end automation test suites with the benefit of getting a detailed timeline of the performance of all the assets (images, javascript files and so on).

Engineers can easily switch between headless and non-headless modes. While writing the automation suite, non-headless mode gives a good view of how the browser is interacting with the application under test and helps in debugging; just like in Selenium. But in production, we can execute suite in a headless mode as it reduces execution time and save resources.

……………………………………………………………………………………………………

Is it really that Good?

Yes, it enjoys the goodness of async code execution. You can always perform multiple tasks simultaneously following the latest javascript async await conventions. You can test the behavior of your application under different screen sizes. The network can be throttled on demand to find out how your application will run at low internet speeds. For e.g. if you want to test the error messages in an application when a user loses internet connectivity, puppeteer can easily handle such a scenario. Mocking APIs can also be achieved with utmost ease. For instance, if you want to test your “payment completed” screen but don’t want to go through the payment gateway flow, you can easily perform it with puppeteer by mocking the API response. With a detailed performance matrix, puppeteer can easily be used for performance testing.

So what are the drawbacks?

Google puppeteer supports only Chromium and Chrome out of the box. Third party implementations are available for Firefox and Internet Explorer. But they are experimental as of now and are missing a good chunk of the firepower. The documentation available is minimal as compared to selenium.

Why should you prefer Puppeteer over Selenium?

The targeted audience is the major deciding factor. If it majorly comprises of Chrome users you should go ahead with Puppeteer. Other factors include performing automation tasks which are not possible with selenium, like testing Chrome extensions, mocking API calls and so on. Selenium follows WebDriver protocol and has been from a very long time in the market compared to puppeteer hence itis better documented compared to puppeteer.

……………………………………………………………………………………………………

Conclusion

Puppeteer is a powerful library which has revolutionized the automation testing by overcoming functionalities missing in WebDriver protocol used by Selenium. Although it is still in the early stages, it promises a bright future, backed up as it is by the illustrious Chrome dev tools team.

……………………………………………………………………………………………………

If you have any queries in this field, talk to Mindfire Solutions. For over 19+ years now, we have been the preferred Software Development Partner of over 1000+ Small and Medium-sized enterprises across the globe.

Spread the love
  • 1
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
    1
    Share

Software Product Development Best Practices

Outsourced Product Development, Software Engineering, Software Industry, Software Product Development, Software Programming, Software QA Testing, ,

Software Product Development Best PracticesThe digital technologies enable enterprises to automate various business processes and operations. But the technological needs differ from one enterprise to another. Based on the technological and business requirements, programmers nowadays have to write a variety of custom software products – desktop applications, web applications and mobile apps. Each type of application is built to meet specific business needs and based on predefined business requirements. Continue reading Software Product Development Best Practices →

Spread the love
  • 6
  • 2
  •  
  •  
  •  
  • 1
  •  
  •  
  •  
    9
    Shares

Best Automation Testing Tools For 2018

Automation Testing Tools, Software QA Testing, , ,

Best Automation Testing Tools For 2018

A steady increase is being noted in the number of enterprises implementing agile methodology to deliver software applications rapidly. Many enterprises adopt DevOps – as an extension of agile methodology – to deliver high quality software applications by unifying software development and IT operations. Both agile methodology and DevOps emphasize on continuous delivery and test automation. The enterprises should coordinate the manual and automation testing efforts to get the best results. Continue reading Best Automation Testing Tools For 2018 →

Spread the love
  • 22
  • 1
  •  
  • 1
  • 5
  •  
  •  
  •  
  •  
    29
    Shares

Benefits of Outsourcing Web Application Testing

QA Testing Outsourcing, Software QA Testing, Web Application Testing, , ,
Benefits of Outsourcing Web Application Testing
Benefits of Outsourcing Web Application Testing

A very large percentage of consumers nowadays access websites and web applications on their mobile devices. But several studies suggest that consumers spend most of their digital media time on specific mobile apps. While developing a web application, an enterprise must focus on its mobile user experience to keep consumers engaged and persuade them to buy. The use experience delivered by a web application depends on a number of factors – accessibility, functionality, usability, performance, and security. Also, the user experience delivered by a single web application differs across browsers, devices, operating systems, and networks. An enterprise must implement a comprehensive web application testing strategy to measure its website’s user experience accurately. Continue reading Benefits of Outsourcing Web Application Testing →

Spread the love
  • 3
  • 3
  •  
  •  
  • 1
  •  
  •  
  •  
  •  
    7
    Shares

Why should you outsource Software QA Testing?

QA Testing Outsourcing, Software QA Testing, ,
Outsource Software QA Testing
Outsource Software QA Testing

Nowadays, consumers access the same software application on varying devices, platforms, and browsers. Your software application must functional flawlessly and deliver richer user experience across different user environments to become profitable. You need to perform elaborate software QA testing to assess the quality and user experience of the software properly. To assess the quality of the software properly, you need to perform various types of software testing – functional testing, performance testing, usability testing, integration testing, acceptance testing, and beta testing. Also, you have to repeat the tests across various devices, platforms, browsers, and networks to generate more reliable test results. Continue reading Why should you outsource Software QA Testing? →

Spread the love
  • 7
  • 2
  •  
  • 2
  •  
  •  
  •  
  •  
  •  
    11
    Shares

Software Testers: Who is your boss?

Software QA Testing, ,

I was spending some time with a software test engineer (Software tester) recently and have a simple realization to share.

“How come this runs so slow?”
“Oh it runs just fine, just some problems only in IE browser”
“Why?”
“Probably IE’s Javascript doesn’t run so well”
“Yes but soooo slow?!?”
“Yes… but my Lead has seen it and knows about it”
“Which means it doesn’t matter?”
“Well not that… client has also agreed to Chrome-only support”
“The site will support only Chrome?”
“Yes, we described the problem and got client to agree to it”
“Do you have it in writing?”
“Yes”
“Very good. But why is this the case in first place?”
“Developers told me the 3rd-party component doesn’t run on IE.” (look on face: how does it matter since client has agreed anyway)
“You mean a best-selling web UI component library doesn’t run well on IE???”
“Yes that’s what developers told me”

After some pushing and research, it finally turned out that developers had to use another method of data access and voila! – everything ran smooth on all browsers.

No doubt, the incident above and its subsequent resolution display several best practices. Get acceptance criteria in writing. Make sure client understands limitations. Don’t trust explanations! Be hard-nosed. Use common sense (how can a best-selling web UI library possibly not function well on IE!). Don’t let your personal love override the obvious (Chrome is great but that doesn’t mean you can force users to switch).

An aspect I would like to focus on is: to whom should the software tester be responsible? Project Manager? Team? CEO? Entity/person who is paying for work?

In the course of project teams and organization and “reporting structure” and salary and revenue and a zillion other things, we forget what is obvious. In reality, the Lead/Manager doesn’t matter. The developers do not matter. The CEO doesn’t matter. The client doesn’t matter! They are all illusion, maya.

The only person who matters, the only person who finally decides, is the user. A software tester is responsible towards only the system being tested, and users of that system.

In the short run, you can get away by keeping your Lead/Manager happy. You can get away by accepting superficial wave-of-hand explanations from web application developers. You can get away by getting clients to agree to something illogical. You can get away by using bureaucracy to cover your behind – “I have it in writing”.

You can get away because the short run is imperfect. The short run can hide. But the long run is all about the right thing. The good thing. The correct thing. Quality. Truth. Details. Finesse. Care. The long run exposes everything.

The right thing here is to find and fix the problem, or find a workaround if it cannot be fixed.

If you do not, all might be well at the surface. But when the user rejects it in the marketplace, everyone loses. And why? Because we ignored her when we should have paid attention. We got confused by irrelevant intermediaries (colleagues, lead, client, CEO) and forgot who the software is being built for.

In the example above, the user is taken for granted. The user is actually expected to tolerate slowness, or switch her browser if she wants to enjoy our system. The IE user votes No and many users are IE-only. Yikes, that pinched!

As a software QA tester (whether for a web site or web application or desktop system or mobile app or server software and so on), you are the user’s voice and warrior. Don’t accept reasons. Make your point. File issues where you see issues. Revolt till someone notices. Help in getting the right thing done. Ride alone if you have to, but ride the right path.

Honestly, you are not alone in getting deceived by quick things. It is the same thing in business. Business gets confused by targets and profits and “customer satisfaction metrics”, because those are the quick and measurable things, although they are not what business is about. Business is about doing the right thing for people, not only the legally right thing.

Software testing is about doing the right thing for users, not only the officially right thing.

Thanks!

Author – Chinmoy Panda

Spread the love
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •