So, What is a White Box Testing?

White Box Testing is the testing of a software solution’s internal coding and infrastructure.

It focuses primarily on strengthening security, the flow of inputs and outputs through the application, design and usability.

White box testing is also known as clear, open, structural, and glass box testing.

It is one of two parts of the “box testing” approach of software testing. Its counter-part, blackbox testing, involves testing from an external or end-user type perspective. On the other hand, Whitebox testing is based on the inner workings of an application and revolves around internal testing. The term “whitebox” was used because of the see-through box concept. The clear box or whitebox name symbolizes the ability to see through the software’s outer shell (or “box”) into its inner workings. Likewise, the “black box” in “black box testing” symbolizes not being able to see the inner workings of the software so that only the end-user experience can be tested

What do you verify in White Box Testing ?

White box testing involves the testing of the software code for the following:

• Internal security holes
• Broken or poorly structured paths in the coding processes
• The flow of specific inputs through the coding infrastructure
• Expected output
• The functionality of conditional loops
• Testing of each statement, object and function on an individual basis

The testing can be done at system, integration and unit levels of software development. One of the basic goals of whitebox testing is to verify a working flow for an application. It involves testing a series of predefined inputs against expected or desired outputs so that when a specific input does not result in the expected output, you have encountered a bug.

How do you perform White Box Testing?

To give you a simplified explanation of white box testing, we have divided it into two basic steps. This is what testers do when testing an application using the white box testing technique:

STEP 1) UNDERSTAND THE SOURCE CODE

The first thing a tester will often do is learn and understand the source code of the application. Since white box testing involves the testing of the inner workings of an application, the tester must be very knowledgeable in the programming languages used in the applications they are testing. Also, the testing person must be highly aware of secure coding practices. Security is often one of the primary objectives of testing software. The tester should be able to find security issues and prevent attacks from hackers and naive users who might inject malicious code into the application either knowingly or unknowingly.

Step 2) CREATE TEST CASES AND EXECUTE

The second basic step to white box testing involves testing the application’s source code for proper flow and structure. One way is by writing more code to test the application’s source code. The tester will develop little tests for each process or series of processes in the application. This  method requires that the tester must have intimate knowledge of the code and is often done by the developer. Other methods include manual testing, trial and error testing and the use of testing tools as we will explain further on in this article.

White Box Testing Techniques

A major White box testing technique is Code Coverage analysis. Code Coverage analysis, eliminates gaps in a test case suite.It identifies areas of a program that are not exercised by a set of test cases.Once gaps are identified, you create test cases to verify untested parts of code, thereby increase the quality of the software product

There are automated tools available to perform Code coverage analysis. Below are a few coverage analysis techniques

Statement Coverage This technique requires every possible statement in the code to be tested at least once during the testing process. Tools: An example of a tool that handles statement coverage for C++ applications is Cantata++

Branch Coverage – This technique checks every possible path (if-else and other conditional loops) of a software application. Tools: An example of a tool that handles branch coverage testing for C, C++ and Java applications is TCAT-PATH

Apart from above, there are numerous coverage types such as Condition Coverage, Multiple Condition Coverage, Path Coverage, Function Coverage etc.Each technique has its own merits and attempts to test (cover) all parts of software code.

Using Statement and Branch coverage you generally attain 80-90% code coverage which is sufficient.

Types of White Box Testing

White box testing encompasses several testing types used to evaluate the usability of an application, block of code or specific software package. There are listed below –

Unit Testing : It is often the first type of testing done on an application. Unit testing is performed on each unit or block of code as it is developed.

Unit testing is essentially done by the programmer. As a software developer, you develop a few lines of code, a single function or an object and test it to make sure it works before continuing

Unit testing helps identify majority of bugs, early in the software development lifecycle. Bugs identified in this stage are cheaper and easy to fix.

Testing for Memory Leaks : Memory leaks are leading causes of slower running applications. A QA specialist who is experienced at detecting memory leaks is essential in cases where you have a slow running software application.

There are many tools available to assist developers/testers with memory leak testing , example , Rational Purify for windows application

Apart from above a few testing types are part of both black box and white box testing. They are listed as below –

White Box Penetration Testing: In this testing, the tester/developer has full information of the application’s source code, detailed network information, IP addresses involved and all server information the application runs on.  The aim is to attack the code from several angles to expose security threats

White Box Mutation Testing: Mutation testing is often used to discover the best coding techniques to use for expanding a software solution .

Ending Notes:

• White box testing can be quite complex. The complexity involved has a lot to do with the application being tested. A small application that performs a single simple operation could be white box tested in minutes, while larger programming applications take days, weeks and even longer to fully test.
• White box testing should be done on a software application as it is being developed, after it is written and again after each modification

A test case describes an input, action, or event and an expected response, to determine if a feature of a software application is working correctly. A test case may contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results. The level of detail may vary significantly depending on the organization and project context.

Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it’s useful to prepare test cases early in the development cycle if possible.

Black box testing – not based on any knowledge of internal design or code. Tests are based on requirements and functionality.
White box testing – based on knowledge of the internal logic of an application’s code. Tests are based on coverage of code statements, branches, paths, conditions.
unit testing – the most ‘micro’ scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses.
incremental integration testing – continuous testing of an application as new functionality is added; requires that various aspects of an application’s functionality be independent enough to work separately before all parts of the program are completed, or that test drivers be developed as needed; done by programmers or by testers.
integration testing – testing of combined parts of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.
functional testing – black-box type testing geared to functional requirements of an application; this type of testing should be done by testers. This doesn’t mean that the programmers shouldn’t check that their code works before releasing it (which of course applies to any stage of testing.)
system testing – black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.
end-to-end testing – similar to system testing; the ‘macro’ end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
sanity testing or smoke testing – typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or corrupting databases, the software may not be in a ‘sane’ enough condition to warrant further testing in its current state.
regression testing – re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing approaches can be especially useful for this type of testing.
acceptance testing – final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.
load testing – testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.
stress testing – term often used interchangeably with ‘load’ and ‘performance’ testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.
performance testing – term often used interchangeably with ‘stress’ and ‘load’ testing. Ideally ‘performance’ testing (and any other ‘type’ of testing) is defined in requirements documentation or QA or Test Plans.
usability testing – testing for ‘user-friendliness’. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.
install/uninstall testing – testing of full, partial, or upgrade install/uninstall processes.
recovery testing – testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
failover testing – typically used interchangeably with ‘recovery testing’
security testing – testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.
compatibility testing – testing how well software performs in a particular hardware/software/operating system/network/etc. environment.
exploratory testing – often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the software as they test it.
ad-hoc testing – similar to exploratory testing, but often taken to mean that the testers have significant understanding of the software before testing it.
context-driven testing – testing driven by an understanding of the environment, culture, and intended use of software. For example, the testing approach for life-critical medical equipment software would be completely different than that for a low-cost computer game.
user acceptance testing – determining if software is satisfactory to an end-user or customer.
comparison testing – comparing software weaknesses and strengths to competing products.
alpha testing – testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.
beta testing – testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.
mutation testing – a method for determining if a set of test data or test cases is useful, by deliberately introducing various code changes (‘bugs’) and retesting with the original test data/cases to determine if the ‘bugs’ are detected. Proper implementation requires large computational resources.

SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.
CMM = ‘Capability Maturity Model’, now called the CMMI (‘Capability Maturity Model Integration’), developed by the SEI. It’s a model of 5 levels of process ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.
Level 1 – characterized by chaos, periodic panics, and heroic
efforts required by individuals to successfully
complete projects. Few if any processes in place;
successes may not be repeatable.

Level 2 – software project tracking, requirements management,
realistic planning, and configuration management
processes are in place; successful practices can
be repeated.

Level 3 – standard software development and maintenance processes
are integrated throughout an organization; a Software
Engineering Process Group is in place to oversee
software processes, and training programs are used to
ensure understanding and compliance.

Level 4 – metrics are used to track productivity, processes,
and products. Project performance is predictable,
and quality is consistently high.

Level 5 – the focus is on continuous process improvement. The
impact of new processes and technologies can be
predicted and effectively implemented when required.

Perspective on CMM ratings: During 1997-2001, 1018 organizations
were assessed. Of those, 27% were rated at Level 1, 39% at 2,
23% at 3, 6% at 4, and 5% at 5. (For ratings during the period
1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and
0.4% at 5.) The median size of organizations was 100 software
engineering/maintenance personnel; 32% of organizations were
U.S. federal contractors or agencies. For those rated at
Level 1, the most problematical key process area was in
Software Quality Assurance.

ISO = ‘International Organisation for Standardization’ – The ISO 9001:2008 standard (which provides some clarifications of the previous standard 9001:2000) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2008 – Quality Management Systems: Requirements; (b)Q9000-2005 – Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2009 – Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products – it indicates only that documented processes are followed. There are also other software-related ISO standards such as ISO/IEC 25010:2011 which includes a ‘quality in use model’ composed of five characteristics and a ‘product quality model’ that covers eight main characteristics of software. Also see http://www.iso.org/ for the latest information. In the U.S. the standards can be purchased via the ASQ web site at http://asq.org/quality-press/
ISO 9126 is a standard for the evaluation of software quality and defines six high level quality characteristics that can be used in software evaluation. It includes functionality, reliability, usability, efficiency, maintainability, and portability.
IEEE = ‘Institute of Electrical and Electronics Engineers’ – among other things, creates standards such as ‘IEEE Standard for Software Test Documentation’ (IEEE/ANSI Standard 829), ‘IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), ‘IEEE Standard for Software Quality Assurance Plans’ (IEEE/ANSI Standard 730), and others.
ANSI = ‘American National Standards Institute’, the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).
Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.
See the Softwareqatest.com ‘Other Resources’ section for further information available on the web.
Return to top of this page’s FAQ list

Web sites are essentially client/server applications – with web servers and ‘browser’ clients. Consideration should be given to the interactions between html pages, TCP/IP communications, Internet connections, firewalls, applications that run in web pages (such as applets, javascript, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.). Additionally, there are a wide variety of servers and browsers, various versions of each, small but sometimes significant differences between them, variations in connection speeds, rapidly changing technologies, and multiple standards and protocols. The end result is that testing for web sites can become a major ongoing effort.

Other considerations might include:
• What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?
• Who is the target audience? What kind of browsers will they be using? What kind of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?
• What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)?
• Will down time for server and content maintenance/upgrades be allowed? how much?
• What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested?
• How reliable are the site’s Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?
• What processes will be required to manage updates to the web site’s content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?
• Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?
• Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??
• How will internal and external links be validated and updated? how often?
• Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet ‘traffic congestion’ problems to be accounted for in testing?
• How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing?
• How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?
Some sources of site security information include the Usenet newsgroup ‘comp.security.announce’ and links concerning web site security in the ‘Other Resources’ section.
Some usability guidelines to consider – these are subjective and may or may not apply to a given situation
• Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page.
• The page layouts and design elements should be consistent throughout a site, so that it’s clear to the user that they’re still within a site.
• Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type.
• All pages should have links external to the page; there should be no dead-end pages.
• The page owner, revision date, and a link to a contact person or organization should be included on each page.
Many new web site test tools have appeared in the recent years and more than 280 of them are listed in the ‘Web Test Tools’ section.

A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the ‘why’ and ‘how’ of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:

• Title
• Identification of software including version/release numbers
• Revision history of document including authors, dates, approvals
• Table of Contents
• Purpose of document, intended audience
• Objective of testing effort
• Software product overview
• Relevant related document list, such as requirements, design documents, other test plans, etc.
• Relevant standards or legal requirements
• Traceability requirements
• Relevant naming conventions and identifier conventions
• Overall software project organization and personnel/contact-info/responsibilities
• Test organization and personnel/contact-info/responsibilities
• Assumptions and dependencies
• Project risk analysis
• Testing priorities and focus
• Scope and limitations of testing
• Test outline – a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable
• Outline of data input equivalence classes, boundary value analysis, error classes
• Test environment – hardware, operating systems, other required software, data configurations, interfaces to other systems
• Test environment validity analysis – differences between the test and production systems and their impact on test validity.
• Test environment setup and configuration issues
• Software migration processes
• Software CM processes
• Test data setup requirements
• Database setup requirements
• Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
• Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs
• Test automation – justification and overview
• Test tools to be used, including versions, patches, etc.
• Test script/test code maintenance processes and version control
• Problem tracking and resolution – tools and processes
• Project test metrics to be used
• Reporting requirements and testing deliverables
• Software entry and exit criteria
• Initial sanity testing period and criteria
• Test suspension and restart criteria
• Personnel allocation
• Personnel pre-training needs
• Test site/location
• Outside test organizations to be utilized and their purpose, responsibilties, deliverables, contact persons, and coordination issues
• Relevant proprietary, classified, security, and licensing issues.
• Open issues
• Appendix – glossary, acronyms, etc.

Many people and organizations are confused about the difference between quality assurance (QA), quality control (QC), and testing. They are closely related, but they are different concepts.

But all these three are useful to manage risks of developing and managing software.

Quality Assurance: A set of activities designed to ensure that the development and/or maintenance process is adequate to ensure a system will meet its objectives.

Quality Control: A set of activities designed to evaluate a developed work product.

Testing: The process of executing a system with the intent of finding defects. (Note that the “process of executing a system” includes test planning prior to the execution of the test cases.)

QA activities ensure that the process is defined and appropriate. Methodology and standards development are examples of QA activities. A QA review would focus on the process elements of a project – e.g., are requirements being defined at the proper level of detail.

QC activities focus on finding defects in specific deliverables – e.g., are the defined requirements the right requirements

Testing is one example of a QC activity, but there are others such as inspections

The difference is that QA is process oriented and QC is product oriented.

Testing therefore is product oriented and thus is in the QC domain. Testing for quality isn’t assuring quality, it’s controlling it.

Quality Assurance makes sure you are doing the right things, the right way.
Quality Control makes sure the results of what you’ve done are what you expected.

So what’s Black Box Testing?

Black box testing is a software testing techniques in which functionality of the software under test (SUT) is tested without looking at the internal code structure, implementation details and knowledge of internal paths of the software. This type of testing is based entirely on the software requirements and specifications.

In Black Box Testing we just focus on inputs and output of the software system without bothering about internal knowledge of the software program.

The above Black Box can be any software system you want to test. For example : an operating system like Windows, a website like Google ,a database like Oracle or even your own custom application. Under Black Box Testing, you can test these applications by just focusing on the inputs and outputs without knowing their internal code implementation.

Black box testing – Steps

Here are the generic steps followed to carry out any type of Black Box Testing.

• Initially requirements and specifications of the system are examined.
• Tester chooses valid inputs (positive test scenario) to check whether SUT processes them correctly. Also some invalid inputs (negative test scenario) are chosen to verify that the SUT is able to detect them.
• Tester determines expected outputs for all those inputs.
• Software tester constructs test cases with the selected inputs.
• The test cases are executed.
• Software tester compares the actual outputs with the expected outputs.
• Defects if any are fixed and re-tested.

Types of Black Box Testing

There are many types of Black Box Testing but following are the prominent ones -

Functional testing – This black box testing type is related to functional requirements of a system; it is done by software testers.

Non-functional testing – This type of black box testing is not related to testing of a specific functionality, but non-functional requirements such as performance, scalability, usability.

Regression testing – Regression testing is done after code fixes , upgrades or any other system maintenance to check the new code has not affected the existing code.

Tools used for Black Box Testing:

Tools used for Black box testing largely depends on the type of black box testing your are doing.

For Functional/ Regression Tests you can use – QTP

For Non-Functional Tests you can use – Loadrunner

Black box testing strategy:

Following are the prominent test strategy amongst the many used in Black box Testing

Equivalence Class Testing: It is used to minimize the number of possible test cases to an optimum level while maintains reasonable test coverage.

Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This technique determines whether a certain range of values are acceptable by the system or not.It is very useful in reducing the number of test cases. It is mostly suitable for the systems where input is within certain ranges.

Decision Table Testing: A decision table puts causes and their effects in a matrix. There is unique combination in each column.

Comparison of Black Box and White Box Testing:

While White Box Testing (Unit Testing) validates internal structure and working of your software code, the main focus of black box testing is on the validation of your functional requirements.

To conduct White Box Testing, knowledge of underlying programming language is essential. Current day software systems use a variety of programming languages and technologies and its not possible to know all of them. Black box testing gives abstraction from code and focuses testing effort on the software system behavior.

Also software systems are not developed in a single chunk but development is broken down in different modules. Black box testing facilitates testing communication amongst modules (Integration Testing) .

In case you push code fixes in your live software system, a complete system check (black box regression tests) becomes essential.

Though White box testing has its own merits and help detect many internal errors which may degrade system performance

Black Box Testing and Software Development Life Cycle (SDLC)

Black box testing has its own life cycle called Software Test Life Cycle (STLC) and it is relative to every stage of Software Development Life Cycle.

Requirement – This is the initial stage of SDLC and in this stage requirement is gathered. Software testers also take part in this stage.

Test Planning  & Analysis – Testing Types applicable to the project are determined. A Test Plan is created which determines possible project risks and their mitigation.

Design – In this stage Test cases/scripts  are created on the basis of software requirement documents

Test Execution- In this stage Test Cases prepared are executed. Bugs if any are fixed and re-tested.

Hope you enjoyed it!

A cookie, also known as an HTTP cookie, web cookie, or browser cookie, is used for an origin website to send state information to a user’s browser and for the browser to return the state information to the origin site. The state information can be used for authentication, identification of a user session, user’s preferences, shopping cart contents, or anything else that can be accomplished through storing text data on the user’s computer.
Cookies cannot be programmed, cannot carry viruses, and cannot install malware on the host computer. However, they can be used by spyware to track user’s browsing activities—a major privacy concern that prompted European and US law makers to take action. Cookie data can also be illicitly disclosed by hackers to gain access to a victim’s web account.

File Transfer Protocol (FTP) is a standard Internet protocol for transmitting files between computers on the Internet. Like the Hypertext Transfer Protocol (HTTP), which transfers displayable Web pages and related files, and the Simple Mail Transfer Protocol (SMTP), which transfers e-mail, FTP is an application protocol that uses the Internet’s TCP/IP protocols. FTP is commonly used to transfer Web page files from their creator to the computer that acts as their server for everyone on the Internet. It’s also commonly used to download programs and other files to your computer from other servers.

As a user, you can use FTP with a simple command line interface (for example, from the Windows MS-DOS Prompt window) or with a commercial program that offers a graphical user interface. Your Web browser can also make FTP requests to download programs you select from a Web page. Using FTP, you can also update (delete, rename, move, and copy) files at a server. You need to logon to an FTP server. However, publicly available files are easily accessed using anonymous FTP.

Basic FTP support is usually provided as part of a suite of programs that come with TCP/IP. However, any FTP client program with a graphical user interface usually must be downloaded from the company that makes it.