Tag: WebAPI

  • Background Job Processing: Just Because You Can Do It in the Request Doesn’t Mean You Should

    Background Job Processing: Just Because You Can Do It in the Request Doesn’t Mean You Should

    Early in my career, I had a simple approach.

    A user submitted a request, and the API did everything before returning a response.

    • Save the data.
    • Send an email.
    • Generate a PDF.
    • Upload files.
    • Notify other systems.

    The endpoint worked.

    But it was also slow.

    Then I realized something important:

    Not every task belongs in the request-response cycle.

    Some tasks don’t need to finish before the user receives a response.

    That’s where background job processing becomes valuable.

    What Is Background Job Processing?

    A background job allows your application to offload long-running or non-critical work to execute after the request has completed.

    Instead of making the user wait, the API responds quickly while the background worker processes the remaining tasks.


    What Should Run in the Background?

    Typical examples include:

    • Sending emails or SMS messages.
    • Generating PDF or Excel reports.
    • Processing uploaded files.
    • Creating thumbnails or resizing images.
    • Synchronizing data with external systems.
    • Publishing events or notifications.

    These operations don’t usually need to block the user’s request.


    Benefits

    • Faster API response times.
    • Better user experience.
    • Improved scalability under heavy load.
    • Better resilience when external services are slow.

    Things to Consider

    Background jobs introduce new responsibilities.

    Think about:

    • Retry policies.
    • Error handling.
    • Monitoring and logging.
    • Idempotency to avoid duplicate processing.
    • Queue management.

    Moving work to the background doesn’t remove complexity—it moves it to a different part of the system.


    Common Tools in .NET

    Depending on your requirements, you might choose:

    • Hangfire
    • Quartz.NET
    • Azure Functions
    • Worker Services
    • Message queues such as RabbitMQ or Azure Service Bus

    The best tool depends on your application’s architecture and operational needs.


    Final Thoughts

    One lesson changed how I design APIs:

    Respond as soon as you’ve completed the work the user actually needs.

    Everything else should be evaluated to see whether it belongs in a background job.

    Fast APIs aren’t always the ones that do less work.

    They’re often the ones that do the right work at the right time.

  • High Concurrency Handling: What Happens When Thousands of Users Hit Your API at the Same Time?

    High Concurrency Handling: What Happens When Thousands of Users Hit Your API at the Same Time?

    Imagine your application is running smoothly.

    Then your company launches a flash sale.

    Or a marketing campaign goes live.

    Suddenly, thousands of users start sending requests to the same endpoint at the same time.

    The question is no longer:

    “Does my API work?”

    It’s:

    “Can my API still work under heavy load?”

    Handling high concurrency isn’t about a single optimization. It’s about building a system that remains reliable as traffic grows.

    Here are some techniques I rely on.


    1. Avoid Blocking Threads

    In ASP.NET Core, every blocked thread reduces your application’s ability to handle incoming requests.

    Prefer asynchronous APIs whenever possible.

    await repository.GetOrdersAsync();
    

    Using async and await allows the thread to serve other requests while waiting for I/O operations.


    2. Optimize Database Access

    The database is often the first bottleneck.

    Reduce unnecessary work by:

    • Selecting only the required columns.
    • Adding proper indexes.
    • Using AsNoTracking() for read-only queries.
    • Avoiding the N+1 query problem.

    A faster query means every request spends less time waiting.


    3. Cache Frequently Requested Data

    Not every request needs to reach the database.

    Cache data that changes infrequently using solutions such as Redis or the in-memory cache.

    Reducing database traffic significantly improves throughput.


    4. Limit Expensive Operations

    Some endpoints perform CPU-intensive or long-running work.

    Protect your application with:

    • Rate limiting.
    • Background processing.
    • Queues for non-critical tasks.

    This prevents a small number of requests from consuming all available resources.


    5. Use Optimistic Concurrency

    When multiple users update the same record simultaneously, conflicts can occur.

    EF Core supports optimistic concurrency using concurrency tokens such as RowVersion.

    This helps prevent accidental overwrites.


    6. Scale Horizontally

    Sometimes optimization isn’t enough.

    Run multiple application instances behind a load balancer so incoming requests are distributed across servers.

    This increases capacity while improving availability.


    7. Monitor Before You Optimize

    Use tools such as:

    • Grafana
    • Prometheus
    • Application Insights
    • OpenTelemetry

    Measure:

    • Response time
    • Error rate
    • CPU usage
    • Database latency
    • Throughput

    You can’t improve what you don’t measure.


    Final Thoughts

    High concurrency isn’t just about surviving traffic spikes.

    It’s about designing systems that continue to respond quickly, remain reliable, and protect shared resources under pressure.

    Performance isn’t achieved through one big optimization.

    It’s the result of many small engineering decisions working together.

  • CORS in ASP.NET Core: It’s More Than Just Fixing a Browser Error

    CORS in ASP.NET Core: It’s More Than Just Fixing a Browser Error

    I’ll admit something.

    For a long time, whenever I saw this error:

    “Access to fetch at ‘https://api.example.com‘ from origin ‘https://localhost:3000‘ has been blocked by CORS policy.”

    My first reaction was simple:

    “Let me add AllowAnyOrigin() and move on.”

    The application worked.

    The error disappeared.

    Problem solved… or so I thought.

    Later, I decided to understand what CORS was actually doing.

    And I realized something important:

    CORS isn’t a server feature. It’s a browser security feature.

    Your API might happily return a response.

    But the browser blocks JavaScript from accessing it if the request violates the server’s CORS policy.

    That’s why Postman works while the browser fails.


    What Is CORS?

    CORS (Cross-Origin Resource Sharing) is a mechanism that allows a server to specify which origins can access its resources.

    An origin consists of:

    • Protocol (https)
    • Domain (example.com)
    • Port (5001)

    If any of these differ, the browser considers it a different origin.

    For example:

    https://localhost:3000
    

    and

    https://localhost:5001
    

    are different origins.


    Configuring CORS in ASP.NET Core

    Register a policy:

    builder.Services.AddCors(options =>
    {
        options.AddPolicy("FrontendPolicy", policy =>
        {
            policy.WithOrigins("https://localhost:3000")
                  .AllowAnyHeader()
                  .AllowAnyMethod();
        });
    });
    

    Apply the policy:

    app.UseCors("FrontendPolicy");
    

    Avoid This in Production

    .AllowAnyOrigin()
    

    While it’s convenient during development, allowing every origin in production can expose your API unnecessarily.

    Instead, explicitly specify the trusted origins your application should allow.


    Middleware Order Matters

    A common mistake is placing UseCors() in the wrong position.

    A typical pipeline looks like this:

    app.UseRouting();
    
    app.UseCors("FrontendPolicy");
    
    app.UseAuthentication();
    
    app.UseAuthorization();
    
    app.MapControllers();
    

    If CORS isn’t applied at the correct point in the pipeline, the browser may still block requests.


    Final Thoughts

    Today, I don’t configure CORS just to remove an error message.

    I configure it knowing why the browser enforces it and how it protects users.

    Understanding the reason behind a feature is far more valuable than memorizing the code to make an error disappear.

  • Lazy Loading vs Eager Loading vs Explicit Loading in EF Core: Which One Should You Use?

    Lazy Loading vs Eager Loading vs Explicit Loading in EF Core: Which One Should You Use?

    When working with Entity Framework Core, loading related data efficiently is just as important as writing correct queries.

    EF Core provides three approaches:

    • Eager Loading
    • Lazy Loading
    • Explicit Loading

    Each has its own strengths and trade-offs.

    Choosing the wrong one can lead to unnecessary database calls and poor application performance.

    Let’s explore when to use each.


    1. Eager Loading

    Eager loading retrieves the main entity and its related data in a single query using Include().

    Example:

    var orders = await context.Orders
        .Include(o => o.Customer)
        .ToListAsync();
    

    Here, both Orders and their related Customer data are loaded together.

    Use Eager Loading When:

    • You know you’ll need the related data.
    • You want to minimize database round trips.
    • You’re building APIs that return complete object graphs.

    Pros

    • Fewer database queries.
    • Better performance in many scenarios.
    • Easy to understand.

    Cons

    • Can retrieve unnecessary data if overused.
    • Multiple Include() calls may produce complex SQL queries.

    2. Lazy Loading

    With lazy loading, related data isn’t loaded until it’s actually accessed.

    Example:

    var order = await context.Orders.FirstAsync();
    
    var customer = order.Customer;
    

    The second line triggers another database query.

    Use Lazy Loading When:

    • Related data is rarely needed.
    • Simplicity is more important than maximum performance.

    Pros

    • Loads data only when needed.
    • Cleaner code.

    Cons

    • Can generate many hidden database queries.
    • May lead to the N+1 query problem.
    • Less predictable performance.

    3. Explicit Loading

    Explicit loading gives you full control over when related data is retrieved.

    Example:

    var order = await context.Orders.FirstAsync();
    
    await context.Entry(order)
        .Reference(o => o.Customer)
        .LoadAsync();
    

    The related entity is loaded only when you explicitly request it.

    Use Explicit Loading When:

    • You only need related data in specific scenarios.
    • You want predictable database access.
    • You’re optimizing performance-critical code.

    Pros

    • Full control over database queries.
    • Avoids unnecessary loading.

    Cons

    • Requires additional code.
    • Easier to forget loading related entities.

    Beware of the N+1 Query Problem

    Consider this:

    var orders = await context.Orders.ToListAsync();
    
    foreach (var order in orders)
    {
        Console.WriteLine(order.Customer.Name);
    }
    

    If lazy loading is enabled, EF Core may execute:

    • 1 query for all orders.
    • 1 additional query for each customer.

    If there are 100 orders, that could result in 101 database queries.

    This is known as the N+1 query problem, and it can significantly impact performance.


    Which One Should You Choose?

    ScenarioRecommended
    You already know you’ll need related data✅ Eager Loading
    Related data is optional and rarely used✅ Lazy Loading
    You need precise control over when data is loaded✅ Explicit Loading

    Final Thoughts

    There isn’t a single “best” loading strategy.

    The right choice depends on your application’s requirements.

    My general rule is:

    • Prefer Eager Loading for APIs and common queries.
    • Use Explicit Loading when you need fine-grained control.
    • Use Lazy Loading carefully, especially in high-performance applications, to avoid hidden database queries.

    Understanding these loading strategies can help you build faster, more efficient EF Core applications.

  • What If a User Uploads a Dangerous File to Your Server? Here’s How to Prevent It.

    What If a User Uploads a Dangerous File to Your Server? Here’s How to Prevent It.

    File upload is one of the most common features in web applications—but it’s also one of the most targeted by attackers.

    Imagine a user uploads a file named:

    invoice.pdf.exe
    

    Or renames a malicious executable to:

    resume.pdf
    

    If your application trusts only the file name or extension, you could expose your server to serious security risks.

    A secure file upload system should never rely on a single validation step.

    1. Validate the File Extension

    Allow only the file types your application actually needs.

    Example:

    • .pdf
    • .jpg
    • .png
    • .docx

    Reject everything else.

    Remember: This is your first line of defense—not your only one.


    2. Verify the MIME Type

    Don’t trust the extension alone.

    Check the file’s MIME type sent by the client.

    For example:

    • application/pdf
    • image/jpeg
    • image/png

    Keep in mind that MIME types can also be spoofed, so continue with additional checks.


    3. Validate the File Signature (Magic Numbers)

    The most reliable validation is checking the file’s binary signature.

    For example:

    • PDF → %PDF
    • PNG → 89 50 4E 47
    • JPEG → FF D8 FF

    If the file signature doesn’t match the expected format, reject the upload.


    4. Limit File Size

    Prevent attackers from uploading extremely large files.

    Example:

    • Images: 5 MB
    • Documents: 20 MB

    This reduces the risk of storage abuse and denial-of-service attacks.


    5. Rename Uploaded Files

    Never store files using the original filename.

    Instead, generate a unique name.

    Example:

    3b8b2d1d-ef7d-4db2-8a8c-0f5d91a0f4a7.pdf
    

    This avoids filename collisions and reduces information disclosure.


    6. Store Files Outside the Web Root

    Avoid storing uploaded files in directories that can execute code.

    Instead:

    • Store files outside the web root.
    • Serve them through a controller or API after authorization.

    This prevents direct execution of uploaded files.


    7. Scan for Malware

    Integrate an antivirus solution to scan uploaded files before making them available.

    This is especially important for systems that accept documents from external users.


    8. Restrict File Permissions

    Uploaded files should never have execute permissions.

    Grant only the minimum permissions required to read or write the file.


    9. Authorize Access

    Not every uploaded file should be publicly accessible.

    Always verify that the requesting user has permission to download or view the file.


    10. Log Upload Activity

    Record important details such as:

    • User ID
    • IP address
    • File name
    • File size
    • Upload time
    • Validation failures

    Logs help detect suspicious activity and support incident investigations.


    Final Thoughts

    Secure file uploads are about defense in depth.

    Don’t rely on a single validation.

    A robust upload pipeline should include:

    • Extension validation
    • MIME type verification
    • File signature checks
    • Size limits
    • Malware scanning
    • Secure storage
    • Proper authorization

    Security isn’t a single feature—it’s a combination of small decisions that work together to protect your application.

  • EF Core Pagination: The Right Way to Handle Large Data Sets

    EF Core Pagination: The Right Way to Handle Large Data Sets

    EF Core Pagination: The Right Way to Handle Large Data Sets

    Returning thousands of records from an API might work during development.

    But what happens when your table has millions of rows?

    This is where pagination becomes essential.


    A common mistake:

    var users = await _context.Users
        .ToListAsync();
    

    It loads everything into memory.

    Problems:

    ❌ Higher memory usage

    ❌ Slower response time

    ❌ Increased database load

    ❌ Poor user experience


    Better Approach: Pagination with EF Core

    var users = await _context.Users
        .OrderBy(x => x.Id)
        .Skip((pageNumber - 1) * pageSize)
        .Take(pageSize)
        .ToListAsync();
    

    How it works:

    Skip()
    ➡️ Ignores previous pages

    Take()
    ➡️ Returns only the required number of records

    Example:

    PageNumber = 3
    PageSize = 20
    

    EF Core skips:

    (3 - 1) × 20 = 40 records
    

    Then returns:

    Next 20 records
    

    Important Pagination Practices

    ✅ Always use OrderBy()

    Without ordering, database results are not guaranteed.

    ✅ Use projection when possible

    Instead of:

    .ToListAsync()
    

    Prefer:

    .Select(x => new UserDto
    {
        Id = x.Id,
        Name = x.Name
    })
    .ToListAsync();
    

    Only fetch the data you need.

    ✅ Return pagination metadata

    Example:

    {
      "pageNumber": 1,
      "pageSize": 20,
      "totalRecords": 500
    }
    

    For Large Tables

    Skip() and Take() work well for most scenarios.

    But for millions of records, consider:

    Keyset Pagination (Seek Pagination)

    It avoids the performance cost of skipping large numbers of rows.


    💡 Rule of thumb:

    Pagination is not just a UI feature.

    It is a database performance strategy.

    A scalable API should control how much data it retrieves and sends.


    👇 What pagination approach do you prefer in your APIs?

    Offset pagination (Skip/Take) or Keyset pagination?

    ♻️ If you found this helpful, feel free to repost and share it with your network.

    👉 Follow Faiz Ahmed Rasel for more .NET tips, tutorials, and deep dives.

    #DotNet #EntityFrameworkCore #CSharp #WebAPI #BackendEngineering