Collection of metrics based on 3 objectives

1.       STATUS: The status of automation (Can be indicated through coverage / execution metrics)
2.       EVALUATION: How automation is performing in the project (ROI and related metrics)
3.       OPPORTUNITIES : Indicate new improvement areas (metrics like lack of coverage, effective automation ect)

This is purely from an automation perspective. Highly recommended when someone throws a bomber like - show me the ROI! or increase this area by x% in automation.

Below are different categorizations of testing that the security team can provide. Based on the application requirements and the timeline different level of security testing options can be chosen.
Level 1 is the most basic which can be performed it the application team has time and only basic testing is required. Level 3 is exhaustive and covers almost all the pages and parameters in the application.

Level 1 Testing

Description: This is the basic testing that we will be performing on application. Before starting the testing we would require pre-requites mentioned above in this document.
Testing Duration: Typically 2 days
Findings Covered: Below are the lists of attacks that will be covered in Level 1 testing on the target application. Based on the scope, only 8-9 high critical pages will be tested and only 6-7 parameters per page will be modified.
  • Cross Site Scripting Stored/Reflected
  • SQL Injection
  • Broken Authentication and Session Management
  • Sensitive data exposer
  • Cross Site Request Forgery
  • Content spoofing
  • Parameter Tampering
  • Forceful Browsing
  • Security Misconfiguration

Level 2 Testing

Description: This is more detailed testing that we will be performing on application. Before starting the testing we would require pre-requites mentioned above in this document.
Testing Duration: Typically 5 days
Findings Covered: Below are the lists of findings/attacks that will be covered in Level 2 testing on the target application. Based on the scope, all high, 8-9 medium and 2-3 low criticality pages will be tested and only 8-9 parameters per page will be modified
  • Cross Site Scripting Stored
  • Cross Site Scripting Reflected
  • SQL Injection
  • Broken Authentication and Session Management
  • Sensitive data exposer
  • Missing Function Level Access
  • Cross Site Request Forgery
  • Content spoofing
  • Hidden Form Fields
  • Insecure Session ID Generation
  • Hijacking Session IDs
  • Parameter Tampering
  • Access to Web Server Directories
  • Default Passwords
  • Account Lockout
  • Forceful Browsing
  • Hidden Form Fields
  • Security Misconfiguration

Level 3 Testing

Description: This is the advanced testing that we will be performing on application. Before starting the testing we would require pre-requites mentioned above in this document.
Testing Duration: Typically 10 days
Findings Covered: Below are the lists of findings/attacks that will be covered in Level 3 testing on the target application. Based on the scope, all the high, medium and low critical pages will be tested and almost all the parameters will be covered.
  • Cross Site Scripting Stored
  • Cross Site Scripting Reflected
  • SQL Injection
  • Broken Authentication and Session Management
  • Sensitive data exposer
  • Missing Function Level Access
  • Cross Site Request Forgery
  • Buffer overflow
  • Content spoofing
  • Hidden Form Fields
  • Information Disclosure
  • Insecure Session ID Generation
  • Hijacking Session IDs
  • Parameter Tampering
  • Access to Web Server Directories
  • Account Lockout
  • Forceful Browsing
  • Information Contained in Cookies
  • Unexpected Input
  • Hidden Form Fields
  • Autocomplete feature set to on
  • SSL Cookie Not In Use
  • Persistent Cookie in use
  • Unvalidated redirects and forwards
  • Older software implementation
  • Default Passwords
  • Security Misconfiguration
A1 – Injection
Injection flaws, such as SQL, OS, and LDAP injection occur when untrusted data is sent to an interpreter as part of a command or query. The attacker’s hostile data can trick the interpreter into executing unintended commands or accessing data without proper authorization.
A2 – Broken Authentication and Session Management
Application functions related to authentication and session management are often not implemented correctly, allowing attackers to compromise passwords, keys, or session tokens, or to exploit other implementation flaws to assume other users’ identities.
A3 – Cross-Site Scripting (XSS)
XSS flaws occur whenever an application takes untrusted data and sends it to a web browser without proper validation or escaping. XSS allows attackers to execute scripts in the victim’s browser which can hijack user sessions, deface web sites, or redirect the user to malicious sites
A4 – Insecure Direct Object References
A direct object reference occurs when a developer exposes a reference to an internal implementation object, such as a file, directory, or database key. Without an access control check or other protection, attackers can manipulate these references to access unauthorized data.
A5 – Security Misconfiguration
Good security requires having a secure configuration defined and deployed for the application, frameworks, application server, web server, database server, and platform. Secure settings should be defined, implemented, and maintained, as defaults are often insecure. Additionally, software should be kept up to date.
A6 – Sensitive Data Exposure
Many web applications do not properly protect sensitive data, such as credit cards, tax IDs, and authentication credentials. Attackers may steal or modify such weakly protected data to conduct credit card fraud, identity theft, or other crimes. Sensitive data deserves extra protection such as encryption at rest or in transit, as well as special precautions when exchanged with the browser.
A7 – Missing Function Level Access Control
Most web applications verify function level access rights before making that functionality visible in the UI. However, applications need to perform the same access control checks on the server when each function is accessed. If requests are not verified, attackers will be able to forge requests in order to access functionality without proper authorization.
A8 - Cross-Site Request Forgery (CSRF)
A CSRF attack forces a logged-on victim’s browser to send a forged HTTP request, including the victim’s session cookie and any other automatically included authentication information, to a vulnerable web application. This allows the attacker to force the victim’s browser to generate requests the vulnerable application thinks are legitimate requests from the victim.
A9 - Using Components with Known Vulnerabilities
Components, such as libraries, frameworks, and other software modules, almost always run with full privileges. If a vulnerable component is exploited, such an attack can facilitate serious data loss or server takeover. Applications using components with known vulnerabilities may undermine application defenses and enable a range of possible attacks and impacts.
A10 – Invalidated Redirects and Forwards

Web applications frequently redirect and forward users to other pages and websites, and use untrusted data to determine the destination pages. Without proper validation, attackers can redirect victims to phishing or malware sites, or use forwards to access unauthorized page.
Here are the pre-requisites from the application team before starting the application security testing.

  • The URL of the application.
  • Valid credentials to access the application (if applicable)
  • Test data
  • Identifying the SPOC (Point of contact from the application team)
  • Meeting with the stakeholders (QA Team Members)
  • To understand the design, functionality and workflow of the application, a walkthrough of the application
There are tons of tools available in the market for security testing, if you know how to use them. Some of the most common/popular ones are below

Static Analysis Tools:
Commercial/Licensed Tools: Checkmarx, HP Fortify, IBM Appscan Source Code, etc
Freeware Tools: FindBugs, FxCrop, Prefast, etc.

Dynamic Analysis Tools:
Commercial/Licensed Tools: HP WebInspect, IBM Appscan Enterprise Edition, Burp Suite Professional, etc
Freeware Tools: Fiddler, ZAP, Paros, WebScarab, Wikto, Nikto, THCsslcheck etc.


Hoping that sometime in future, I will be able to add some video tutorials.
Application Security testing focuses on identifying application and configuration vulnerabilities that could lead to security issues. The goal of the reviews is to identify as many potential security vulnerabilities as possible. Application Security Testing can be performed using both or anyone of the below mentioned approaches/methods:
·         Tool Based Scan/Vulnerability Scan:  In the tool based scan or vulnerability scan a tool is used to perform the security checks on the application.
·         Manual Security Scan/Penetration testing : The penetration testing or manual security scan is a method where the application is exploited manually with the use of proxy tools. Findings flaws in the design and business logic.

Further, there are two ways to perform the Application Security Testing: - Static and Dynamic. Depending on the stage of the application in SDLC cycle, any one way can be used.

·         Static Analysis: Static Analysis also termed as Static application security testing (SAST) can be thought of as testing the application from the inside out – by examining its source code, byte code or application binaries for conditions indicative of a security vulnerability.

·         Dynamic Analysis: Dynamic Analysis also termed as Dynamic application security testing (DAST) can be thought of as testing the application from the outside in – by examining the application in its running state and trying to poke it and prod it in unexpected ways in order to discover security vulnerabilities
Open Source Tools
​Test Management Tools Functional Testing Tools Load Testing Tools
Commercial Tools
Test Management Tools Functional Testing Tools Load Testing Tools
A mind map for companies and users to select test tools for evaluation.
Tool Evaluation - Mind Map

Showing key stakeholders that your test automation efforts are worth continued investment is often tricky, because value is a relative term that varies based on a given organization's needs. One important aspect in relaying value is telling a well-rounded story of how test automation has been beneficial to your organization. This story might include the following:
 
·         Script Development Metrics - Identifying metrics related to the percentage of manual tests that have been automated helps to reveal the productivity of your team
·         Cost Savings ROI - The amount of money that has been saved in the test implementation
·         Efficiency ROI - The amount of time that has been saved in the test implementation
·         Risk Reduction ROI - The amount of money that has potentially been saved by reducing risks, increasing test coverage, and allowing manual testers to focus on finding costly, hard to find defects
·         Defects Identified - The number of defects found during automated test development and/or implementation
·         Organizational Impact - Revealing how the introduction of automation has caused testing to be done earlier, or how it has caused development and/or testing practices to improve, or how some small utilities you've created are being used by other teams to increase their efficiencies


The Business value articulation of this problem is the most sought after deliverable.
top