Technical Details
This page is a technical showcase of various issues that Schemathesis.io can detect in your API.
It includes root cause details, code snippets, and detailed API specifications.
The content here is intended for a technical audience, focusing on the specifics and intricacies of each case.
For access to the API and its schema, visit https://example.schemathesis.io/openapi.yaml.
If you are looking for a more generalized overview, a non-technical version of this documentation is also available.
🔒 Security​
Vulnerabilities that can compromise your API's security, leading to unauthorized data access and other security breaches.
Unauthenticated Access​
GET /security/unauthenticated-access
Endpoints declared to require authentication in the API specification must enforce this in the implementation to prevent unauthorized access to sensitive data.
In this case, the developer overlooked the addition of an authentication check to a web handler, potentially exposing sensitive data to unauthorized users.
# Uncomment and configure to enforce authentication
# from flask_security import auth_required
from flask import Flask
app = Flask(__name__)
@app.route("/security/unauthenticated-access", methods=["GET"])
# Protect your endpoint by uncommenting this line
# @auth_required('token', 'session')
def unauthenticated_access():
return {"data": "Sensitive"}
API Specification
security:
- ApiKeyAuth: []
responses:
'200':
description: Success
content:
application/json:
schema:
type: object
properties:
data:
type: string
required:
- data
'401':
description: Unauthorized
content:
application/json:
schema:
type: object
properties:
error:
type: string
example:
error: Unauthorized access
💥 Internal Server Errors​
A selection of real-life project-based internal server errors. This section emphasizes the data generation and test execution strategies Schemathesis.io utilizes to identify input-related server errors.
Improper Input Type Handling​
POST /internal-server-errors/improper-input-type-handling
An issue originating from improper input validation, allowing non-numeric inputs to cause a server error during the card number validation process.
In this example, the server errors when it attempts to convert non-numeric characters to integers, which is not properly handled in the card number validation process.
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.route("/internal-server-errors/improper-input-type-handling", methods=["POST"])
def improper_input_type_handling():
data = request.json
if not isinstance(data, dict) or "number" not in data:
return jsonify({"success": False}), 400
# Potential crash point
digits = [int(d) for d in str(data["number"])]
# Luhn algorithm to validate card number
even_digits_sum = sum(digits[-1::-2])
odd_digits_sum = sum(sum(divmod(d * 2, 10)) for d in digits[-2::-2])
checksum = even_digits_sum + odd_digits_sum
is_valid = checksum % 10 == 0
return jsonify({"success": is_valid})
API Specification
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
number:
type: string
description: The credit card number to validate.
example: '1234567812345670'
required:
- number
responses:
default:
description: Card number validation result
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Indicates whether the card number is valid.
error:
type: string
description: Error text if any.
required:
- success
Exceeding Column Size​
POST /internal-server-errors/exceeding-column-size
A server error can occur when attempting to store an input exceeding the database column's restricted size, resulting in a database error.
In this scenario, an attempt to store input text that exceeds the 255 character limit of the database column results in a database error and server failure.
import sqlalchemy.exc
from flask import Flask, jsonify, request
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite://"
db = SQLAlchemy(app)
class Message(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Column with restricted size
text = db.Column(db.String(16), sqlalchemy.CheckConstraint("LENGTH(text) < 16"))
@app.route("/internal-server-errors/exceeding-column-size", methods=["POST"])
def exceeding_column_size():
data = request.json
if "text" not in data:
return jsonify({"success": False}), 400
# Storing input that exceeds the column size limit can result in a database error
message = Message(text=data["text"])
db.session.add(message)
try:
db.session.commit()
return jsonify({"success": True})
except sqlalchemy.exc.SQLAlchemyError as exc:
db.session.rollback()
return jsonify({"success": False, "error": str(exc)}), 500
API Specification
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
text:
type: string
description: The input text to be stored in the database.
required:
- text
additionalProperties: false
responses:
'200':
description: Successfully stored the input text in the database.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Indicates whether the text was successfully stored in the
database.
example: true
'400':
description: Bad Request, missing 'text' field in the request body.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Always false for this response.
example: false
'500':
description: Internal Server Error due to a database error.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Always false for this response.
example: false
error:
type: string
description: Error message indicating a database error.
example: Database error
Improper Unicode Encoding​
POST /internal-server-errors/improper-unicode-encoding
The section highlights issues stemming from the improper handling of Unicode inputs, including emojis and other non-standard text elements.
Here, the improper handling of a text input containing Unicode characters during an attempt to encode it to ASCII format triggers a UnicodeDecodeError.
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.route("/internal-server-errors/improper-unicode-encoding", methods=["POST"])
def improper_unicode_encoding():
data = request.json
if "text" not in data:
return jsonify({"success": False, "error": "Missing text field"}), 400
try:
# Simulating improper Unicode handling
data["text"].encode("ascii")
return jsonify({"success": True})
except UnicodeDecodeError:
return jsonify({"success": False, "error": "Unicode handling error"}), 500
API Specification
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
text:
type: string
description: The input text to be encoded to ASCII.
required:
- text
responses:
'200':
description: Successfully encoded the text to ASCII.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Indicates whether the text encoding was successful.
example: true
'400':
description: Bad Request, missing 'text' field in the request body.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Always false for this response.
example: false
error:
type: string
description: Error message indicating the missing 'text' field.
example: Missing text field
'500':
description: Internal Server Error due to Unicode handling error.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Always false for this response.
example: false
error:
type: string
description: Error message indicating Unicode handling error.
example: Unicode handling error
📋 Response Conformance​
This section highlights discrepancies between the actual API responses and the structures defined in the specifications.
Malformed Json​
GET /response-conformance/malformed-json
An issue where the server responds with unstructured error messages instead of the expected structured JSON format, leading to potential confusion and improper handling of the response.
In this scenario, the server returns a malformed JSON string due to a typo while bypassing regular JSON serialization for perceived simplicity. This could lead to errors in clients using standard JSON parsers to interpret the response.
from flask import Flask
app = Flask(__name__)
@app.route("/response-conformance/malformed-json", methods=["GET"])
def malformed_json():
return "{success: true}", 200, {"Content-Type": "application/json"}
API Specification
responses:
default:
description: Default response.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
description: Indicates whether response is successful.
required:
- success
Incorrect Content Type​
GET /response-conformance/incorrect-content-type
Returning a response with a content type that differs from what is specified in the API schema, leading to potential client-side issues as the clients might be expecting a different data format.
In this scenario, when the item_id parameter is set to "error", the server returns a plain text response, violating the API schema and potentially causing bugs and failures in client applications.
from flask import Flask
app = Flask(__name__)
@app.route("/response-conformance/incorrect-content-type", methods=["GET"])
def incorrect_content_type():
# This response does not conform to the OpenAPI schema
return "Success!", 200, {"Content-Type": "text/plain"}
API Specification
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
name:
type: string
Missing Field​
GET /response-conformance/missing-field
Returning a response that misses some fields specified in the API schema. This discrepancy can lead to integration issues, as clients expecting the missing field might encounter errors or incorrect behavior.
In this instance, the "age" field, as defined in the API schema, is absent from the response, which might result in errors or unexpected behavior in client applications.
from flask import Flask, jsonify
app = Flask(__name__)
@app.route("/response-conformance/missing-field", methods=["GET"])
def missing_field():
response_data = {
"id": "123",
"name": "Alice"
# "age" field is missing
}
return jsonify(response_data), 200
API Specification
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
id:
type: string
name:
type: string
age:
type: integer
required:
- id
- name
- age
Undocumented Status Code​
GET /response-conformance/undocumented-status-code
An issue where the server responds with a status code that is not documented in the OpenAPI schema.
In this scenario, the API endpoint can potentially return a 404 status code when the requested ID does not exist in the database, leading to unexpected behaviors in client applications.
from flask import Flask, jsonify, request
app = Flask(__name__)
# Simulating a database with a dictionary
data_db = {"0": "Data for ID 0"}
@app.route("/response-conformance/undocumented-status-code", methods=["GET"])
def undocumented_status_code():
id = request.args.get("id")
if id is None:
return jsonify({"error": "ID is required"}), 400
data = data_db.get(str(id))
if data is None:
# Returning a 404 status code, which is not documented in the API schema
return jsonify({"error": "Not Found"}), 404
return jsonify({"message": data})
API Specification
parameters:
- name: id
in: query
description: The ID of the item to fetch
required: true
schema:
type: integer
responses:
'200':
description: Success
content:
application/json:
schema:
type: object
properties:
message:
type: string
required:
- message
'400':
description: Invalid input.
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: A descriptive error message indicating the input validation
failure.
🚀 Performance​
Uncovers issues that affect the API’s performance, including inefficient data handling and the impact of specific input patterns on response times.
Unbounded Result Set​
GET /performance/unbounded-result-set
Endpoints returning extensive, unbounded result sets can significantly strain server resources, leading to performance degradation and potentially rendering the service unresponsive.
This case illustrates an endpoint that allows clients to request a large number of items without proper limitations. Implementing pagination or enforcing limitations on result set sizes is essential to prevent server strain, maintain optimal performance, and keep the service responsive.
import time
from flask import Flask, jsonify, request
app = Flask(__name__)
# Maximum number of items that can be fetched in one request
MAX_ITEMS = 1000
# Simulated average delay per item retrieval in seconds
DELAY_PER_ITEM = 0.001 # 1 ms
@app.route("/performance/unbounded-result-set", methods=["GET"])
def unbounded_result_set():
limit = min(request.args.get("limit", default=MAX_ITEMS, type=int), MAX_ITEMS)
if limit <= 0:
return jsonify({"error": "Limit must be greater than 0"}), 400
# Simulate fetching 'limit' number of items with a delay for each item retrieval.
# The delay simulates the additional time required for transferring more data.
items = {}
for idx in range(limit):
# Simulate the delay for retrieving each item
time.sleep(DELAY_PER_ITEM)
# Simulate the generation of an item
items[f"item_{idx}"] = "data"
# It's crucial to limit the number of items fetched in one request to prevent server strain and
# maintain optimal performance.
return jsonify(items)
API Specification
parameters:
- name: limit
in: query
description: The number of items to fetch. Must be greater than 0 and not exceed
the maximum limit.
required: true
schema:
type: integer
default: 1000
maximum: 1000
minimum: 1
responses:
'200':
description: Successfully fetched items.
content:
application/json:
schema:
type: object
additionalProperties:
type: string
description: The data for each item.
'400':
description: Bad Request. Limit must be greater than 0.
content:
application/json:
schema:
type: object
properties:
error:
type: string
example: Limit must be greater than 0
Inefficient Algorithm​
GET /performance/inefficient-algorithm
Inefficient algorithms can significantly slow down API responses, especially when handling large input, leading to poor performance and potential timeouts.
In this scenario, generating a large Fibonacci sequence using an inefficient algorithm results in slow response times. The impact is particularly noticeable when searching for a term within the sequence, as the delay compounds, potentially leading to timeouts and degraded user experience. Setting reasonable limits on input sizes and optimizing algorithms are essential to mitigate such performance issues.
from flask import Flask, jsonify, request
app = Flask(__name__)
# Set a reasonable limit for the maximum number of Fibonacci numbers to generate
MAX_N = 100000
def generate_fibonacci(n):
# The loop generates Fibonacci numbers inefficiently, leading to increased response times for large n.
fib_sequence = [0, 1]
while len(fib_sequence) < n:
fib_sequence.append(fib_sequence[-1] + fib_sequence[-2])
return fib_sequence
@app.route("/performance/inefficient-algorithm", methods=["GET"])
def inefficient_algorithm():
n = request.args.get("n", type=int)
search_term = request.args.get("searchTerm", type=int)
if n is None or search_term is None:
return jsonify({"error": "Missing required parameters"}), 400
if n > MAX_N:
return jsonify({"error": f"n should be less than or equal to {MAX_N}"}), 400
# Generating a large Fibonacci sequence
fib_sequence = generate_fibonacci(n)
# Searching for the term in the sequence
found_indices = [
index for index, value in enumerate(fib_sequence) if value == search_term
]
return jsonify({"foundAt": found_indices})
API Specification
parameters:
- name: n
in: query
description: The length of the Fibonacci sequence to be generated.
required: true
schema:
type: integer
minimum: 1
maximum: 100000
- name: searchTerm
in: query
description: The term to search for within the generated Fibonacci sequence.
required: true
schema:
type: integer
minimum: 0
responses:
'200':
description: Successfully found the search term in the Fibonacci sequence.
content:
application/json:
schema:
type: object
properties:
foundAt:
type: array
items:
type: integer
description: The indices at which the search term was found in the Fibonacci
sequence.
'400':
description: Invalid input. The 'n' should be less than or equal to 30 and 'searchTerm'
must be a non-negative integer.
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: A descriptive error message indicating the input validation
failure.