Project: CSV Import Worker Service (C# Tutorial)

In this project you build the kind of program that runs quietly in thousands of companies: a background service that watches a folder, imports files dropped there by another system, and reports what it found. Here, shops in Jaipur, Pune and Kochi export their daily sales from their billing software as CSV files. Your service picks up each file, validates every line, writes a JSON summary with totals per city, and moves the original file to an archive folder.

The project brings together the professional topics of this course: a solution with separate projects (Solutions, Projects & NuGet Packages), the Generic Host with options and logging (Dependency Injection, Configuration & Logging), async file I/O and cancellation, System.Text.Json, and unit tests. Build it step by step and run the tests after each part.

The plan

Item	Decision
Input	`*.csv` files in an inbox folder, header `OrderId,City,Item,Qty,UnitPrice`
Valid line	5 fields, an order ID, `Qty` a whole number of at least 1, `UnitPrice` a number of 0 or more
Output	`<name>.summary.json` in an outbox folder: counts, total, totals per city, errors with line numbers
After import	The CSV moves to an archive folder
Bad file access	Log the error and retry on the next check
Check interval	Configurable, 5 seconds while developing

The solution has three projects, and the most important rule is the same as in the library manager project: business logic does not know it is running in a worker.

OrderImport.Core: a class library with the parser and the file processor.
OrderImport.Worker: the background service that calls the library on a timer.
OrderImport.Tests: xUnit tests for the library.

Step 1: create the solution

Terminal

mkdir OrderImport
cd OrderImport
dotnet new sln
dotnet new gitignore
dotnet new classlib -o src/OrderImport.Core
dotnet new worker -o src/OrderImport.Worker
dotnet new xunit3 -o tests/OrderImport.Tests
dotnet sln add src/OrderImport.Core/OrderImport.Core.csproj src/OrderImport.Worker/OrderImport.Worker.csproj tests/OrderImport.Tests/OrderImport.Tests.csproj
dotnet add src/OrderImport.Worker/OrderImport.Worker.csproj reference src/OrderImport.Core/OrderImport.Core.csproj
dotnet add tests/OrderImport.Tests/OrderImport.Tests.csproj reference src/OrderImport.Core/OrderImport.Core.csproj
dotnet add src/OrderImport.Worker/OrderImport.Worker.csproj package Microsoft.Extensions.Options.DataAnnotations

The worker template creates a project that uses the Worker SDK, an appsettings.json, a Program.cs with the Generic Host and a sample Worker class. If you installed the xUnit v3 templates for the unit testing lesson, xunit3 is ready; otherwise run dotnet new install xunit.v3.templates first. Delete Class1.cs from the library and UnitTest1.cs from the tests.

Step 2: the model and the parser

In src/OrderImport.Core, create Parsing.cs:

using System.Globalization;

namespace OrderImport.Core;

public sealed record OrderLine(string OrderId, string City, string Item, int Qty, decimal UnitPrice)
{
    public decimal Amount => Qty * UnitPrice;
}

public sealed class ParseResult
{
    public List<OrderLine> Orders { get; } = [];
    public List<string> Errors { get; } = [];
}

public static class CsvOrderParser
{
    public static ParseResult Parse(IEnumerable<string> lines)
    {
        var result = new ParseResult();
        int lineNumber = 0;

        foreach (string raw in lines)
        {
            lineNumber++;
            if (lineNumber == 1 || string.IsNullOrWhiteSpace(raw))
            {
                continue; // skip the header and blank lines
            }

            string? error = TryParseLine(raw, lineNumber, out OrderLine? order);
            if (order is not null)
            {
                result.Orders.Add(order);
            }
            else
            {
                result.Errors.Add(error!);
            }
        }
        return result;
    }

    private static string? TryParseLine(string raw, int n, out OrderLine? order)
    {
        order = null;
        string[] f = raw.Split(',', StringSplitOptions.TrimEntries);

        if (f.Length != 5)
            return $"Line {n}: expected 5 fields but found {f.Length}";
        if (f[0].Length == 0)
            return $"Line {n}: OrderId is missing";
        if (!int.TryParse(f[3], NumberStyles.Integer, CultureInfo.InvariantCulture, out int qty))
            return $"Line {n}: Qty {f[3]} is not a whole number";
        if (qty < 1)
            return $"Line {n}: Qty must be at least 1";
        if (!decimal.TryParse(f[4], NumberStyles.Number, CultureInfo.InvariantCulture, out decimal price) || price < 0)
            return $"Line {n}: UnitPrice {f[4]} is not valid";

        order = new OrderLine(f[0], f[1], f[2], qty, price);
        return null;
    }
}

Design choices worth noticing:

The parser takes IEnumerable<string>, not a file path. Tests can pass an array of strings, and the worker can pass lines read from disk.
A bad line does not stop the import. Real data is messy; the summary lists every problem with its line number so the shop can fix it.
CultureInfo.InvariantCulture means 1250.50 is read the same way on every server, whatever its regional settings.
StringSplitOptions.TrimEntries removes stray spaces around fields. This simple Split does not handle quoted fields containing commas; for full CSV support a library such as CsvHelper is the usual choice.

Step 3: the file processor

Create ImportProcessor.cs in the same project. It reads one file, builds a summary, writes it as JSON and archives the CSV.

using System.Text.Json;

namespace OrderImport.Core;

public sealed record ImportSummary(
    string File,
    int ValidOrders,
    int InvalidLines,
    decimal Total,
    SortedDictionary<string, decimal> CityTotals,
    List<string> Errors);

public static class ImportProcessor
{
    private static readonly JsonSerializerOptions Json =
        new(JsonSerializerDefaults.Web) { WriteIndented = true };

    public static async Task<ImportSummary> ProcessFileAsync(
        string csvPath, string outboxFolder, string archiveFolder, CancellationToken ct)
    {
        string[] lines = await File.ReadAllLinesAsync(csvPath, ct);
        ParseResult parsed = CsvOrderParser.Parse(lines);

        var cityTotals = new SortedDictionary<string, decimal>(StringComparer.Ordinal);
        foreach (OrderLine order in parsed.Orders)
        {
            cityTotals.TryGetValue(order.City, out decimal soFar);
            cityTotals[order.City] = soFar + order.Amount;
        }

        string fileName = Path.GetFileName(csvPath);
        var summary = new ImportSummary(fileName, parsed.Orders.Count, parsed.Errors.Count,
            parsed.Orders.Sum(o => o.Amount), cityTotals, parsed.Errors);

        Directory.CreateDirectory(outboxFolder);
        Directory.CreateDirectory(archiveFolder);

        string summaryPath = Path.Combine(outboxFolder, Path.ChangeExtension(fileName, ".summary.json"));
        await using (FileStream stream = File.Create(summaryPath))
        {
            await JsonSerializer.SerializeAsync(stream, summary, Json, ct);
        }

        File.Move(csvPath, Path.Combine(archiveFolder, fileName), overwrite: true);
        return summary;
    }
}

The SortedDictionary keeps cities in alphabetical order, so the JSON is the same every time, which makes files easy to compare. The options object is created once in a static readonly field, as recommended in System.Text.Json in Depth. The summary is written before the CSV is moved: if the service crashes in between, the file is simply imported again on restart rather than lost.

Step 4: test the library

Now prove the logic works before any worker exists. In tests/OrderImport.Tests, create CsvOrderParserTests.cs:

using OrderImport.Core;

namespace OrderImport.Tests;

public class CsvOrderParserTests
{
    [Fact]
    public void Parse_KeepsValidLines_AndReportsBadOnes()
    {
        string[] lines =
        [
            "OrderId,City,Item,Qty,UnitPrice",
            "A101,Jaipur,Kurta,2,799",
            "A103,Jaipur,Dupatta,abc,299",
            "",
            "A105,Pune,Mixer,0,3199"
        ];

        ParseResult result = CsvOrderParser.Parse(lines);

        OrderLine order = Assert.Single(result.Orders);
        Assert.Equal(1598m, order.Amount);
        Assert.Equal(2, result.Errors.Count);
        Assert.Equal("Line 3: Qty abc is not a whole number", result.Errors[0]);
        Assert.Equal("Line 5: Qty must be at least 1", result.Errors[1]);
    }

    [Theory]
    [InlineData("A1,Pune,Pen,1", "Line 2: expected 5 fields but found 4")]
    [InlineData(",Pune,Pen,1,10", "Line 2: OrderId is missing")]
    [InlineData("A1,Pune,Pen,1,-5", "Line 2: UnitPrice -5 is not valid")]
    public void Parse_BadLine_ExplainsWhy(string line, string expected)
    {
        ParseResult result = CsvOrderParser.Parse(["OrderId,City,Item,Qty,UnitPrice", line]);

        Assert.Empty(result.Orders);
        Assert.Equal(expected, Assert.Single(result.Errors));
    }
}

Then ImportProcessorTests.cs, which uses a real temporary folder and deletes it afterwards:

using OrderImport.Core;

namespace OrderImport.Tests;

public sealed class ImportProcessorTests : IDisposable
{
    private readonly string _root = Directory.CreateTempSubdirectory("order-import-").FullName;

    [Fact]
    public async Task ProcessFile_WritesSummary_AndArchivesCsv()
    {
        CancellationToken ct = TestContext.Current.CancellationToken;
        string inbox = Path.Combine(_root, "inbox");
        Directory.CreateDirectory(inbox);
        string csv = Path.Combine(inbox, "shop-1.csv");
        await File.WriteAllLinesAsync(csv,
            ["OrderId,City,Item,Qty,UnitPrice", "B1,Kochi,Tawa,3,449", "B2,Kochi,Kadai,1,650"], ct);

        ImportSummary summary = await ImportProcessor.ProcessFileAsync(
            csv, Path.Combine(_root, "outbox"), Path.Combine(_root, "archive"), ct);

        Assert.Equal(1997m, summary.Total);
        Assert.Equal(1997m, summary.CityTotals["Kochi"]);
        Assert.False(File.Exists(csv));
        Assert.True(File.Exists(Path.Combine(_root, "archive", "shop-1.csv")));
        Assert.True(File.Exists(Path.Combine(_root, "outbox", "shop-1.summary.json")));
    }

    public void Dispose() => Directory.Delete(_root, recursive: true);
}

Run dotnet test from the solution folder. All five tests (one fact and three theory rows for the parser, one fact for the processor) should pass. xUnit calls Dispose after each test, so no temporary folders are left behind. Kochi's total is 3 x 449 + 650 = 1997.

Step 5: the options and the worker

Now the service. In src/OrderImport.Worker, delete Worker.cs and create ImportWorker.cs:

using System.ComponentModel.DataAnnotations;
using Microsoft.Extensions.Options;
using OrderImport.Core;

namespace OrderImport.Worker;

public sealed class ImportOptions
{
    public const string Section = "Import";

    [Required] public string InboxFolder { get; set; } = "";
    [Required] public string OutboxFolder { get; set; } = "";
    [Required] public string ArchiveFolder { get; set; } = "";
    [Range(1, 3600)] public int PollSeconds { get; set; } = 10;
}

public sealed class ImportWorker(
    IOptions<ImportOptions> options,
    ILogger<ImportWorker> logger,
    TimeProvider clock) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        ImportOptions o = options.Value;
        Directory.CreateDirectory(o.InboxFolder);
        logger.LogInformation("Watching {Inbox} every {Seconds} s", o.InboxFolder, o.PollSeconds);

        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(o.PollSeconds), clock);
        do
        {
            await ImportWaitingFilesAsync(o, stoppingToken);
        }
        while (await timer.WaitForNextTickAsync(stoppingToken));
    }

    private async Task ImportWaitingFilesAsync(ImportOptions o, CancellationToken ct)
    {
        // Order() reads the whole list first, so moving files cannot disturb the loop.
        foreach (string path in Directory.EnumerateFiles(o.InboxFolder, "*.csv").Order())
        {
            try
            {
                ImportSummary s = await ImportProcessor.ProcessFileAsync(
                    path, o.OutboxFolder, o.ArchiveFolder, ct);
                logger.LogInformation("Imported {File}: {Valid} orders, {Invalid} bad lines, total {Total}",
                    s.File, s.ValidOrders, s.InvalidLines, s.Total);
            }
            catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
            {
                logger.LogError(ex, "Could not import {File}; will retry on the next check",
                    Path.GetFileName(path));
            }
        }
    }
}

How it works:

BackgroundService is the base class for long-running work. The host calls ExecuteAsync once at start-up and cancels stoppingToken when the service must stop (Ctrl+C, a Windows service stop, or docker stop).
PeriodicTimer (with a TimeProvider, so tests could use FakeTimeProvider) ticks every few seconds. WaitForNextTickAsync waits without blocking a thread and throws OperationCanceledException on shutdown, which the host treats as a normal stop.
The do ... while shape imports immediately on start-up, then on every tick.
Only file-access errors are caught: a CSV that the billing software is still writing is locked, so the worker logs it and tries again later. Any other exception is a bug and should not be hidden.

Step 6: wire up Program.cs and the settings

Replace src/OrderImport.Worker/Program.cs:

using OrderImport.Worker;

HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);

builder.Services
    .AddOptions<ImportOptions>()
    .Bind(builder.Configuration.GetSection(ImportOptions.Section))
    .ValidateDataAnnotations()
    .ValidateOnStart();

builder.Services.AddSingleton(TimeProvider.System);
builder.Services.AddHostedService<ImportWorker>();

IHost host = builder.Build();
host.Run();

Add an Import section to the existing appsettings.json:

JSON

{
  "Import": {
    "InboxFolder": "data/inbox",
    "OutboxFolder": "data/outbox",
    "ArchiveFolder": "data/archive",
    "PollSeconds": 5
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

ValidateOnStart checks the settings as the host starts, so a missing folder name stops the service immediately with a clear OptionsValidationException instead of failing hours later. AddHostedService registers the worker as a singleton that the host starts and stops. Relative folder paths are resolved from the current folder; on a server you would set absolute paths through environment variables such as Import__InboxFolder.

Step 7: run it

Start the service from the worker folder:

Terminal

cd src/OrderImport.Worker
dotnet run

While it runs, save this file as data/inbox/sales-2026-11-02.csv inside the worker folder:

Text

OrderId,City,Item,Qty,UnitPrice
A101,Jaipur,Kurta,2,799
A102,Pune,Saree,1,2499
A103,Jaipur,Dupatta,abc,299
A104,Kochi,Tawa,3,449
A105,Pune,Mixer,0,3199

Within five seconds the worker logs something like this (the host's own start-up messages also appear in between):

Output

info: OrderImport.Worker.ImportWorker[0]
      Watching data/inbox every 5 s
info: OrderImport.Worker.ImportWorker[0]
      Imported sales-2026-11-02.csv: 3 orders, 2 bad lines, total 5444

The CSV is now in data/archive, and data/outbox/sales-2026-11-02.summary.json contains:

JSON

{
  "file": "sales-2026-11-02.csv",
  "validOrders": 3,
  "invalidLines": 2,
  "total": 5444,
  "cityTotals": {
    "Jaipur": 1598,
    "Kochi": 1347,
    "Pune": 2499
  },
  "errors": [
    "Line 4: Qty abc is not a whole number",
    "Line 6: Qty must be at least 1"
  ]
}

Check the numbers: Jaipur 2 x 799 = 1598, Pune 2499 (the Mixer line has quantity 0 and is rejected), Kochi 3 x 449 = 1347, total 5444. Property names are camelCase because of the web defaults, while dictionary keys such as city names are written unchanged. Press Ctrl+C and the host stops the worker cleanly.

Step 8: run it as a real service

A worker usually runs unattended, started by the operating system:

Windows: add the Microsoft.Extensions.Hosting.WindowsServices package and call builder.Services.AddWindowsService(o => o.ServiceName = "Order Import"); in Program.cs. Publish the app and register the .exe as a Windows service, for example with the sc.exe create command.
Linux: add Microsoft.Extensions.Hosting.Systemd, call builder.Services.AddSystemd();, publish for linux-x64, and describe the service in a systemd unit file.
Containers: a worker runs well in Docker or Kubernetes; the host already handles the SIGTERM signal that docker stop sends.

Both helper methods do nothing when the app is started normally from a terminal, so the same build still works with dotnet run.

Ideas to take it further

Add a test for ImportWorker itself: give it a FakeTimeProvider and a temporary inbox, start it with StartAsync, advance the clock, and check the outbox.
Reject a file that has already been imported (same name in the archive) instead of overwriting it, and log a warning.
Read files with File.ReadLinesAsync and parse line by line so very large files never sit in memory at once.
Process several files in parallel with Parallel.ForEachAsync, or hand file paths to a Channel<string> read by a fixed number of consumers, as in Async Patterns: Cancellation, ValueTask & Channels.
Save summaries to a database with EF Core instead of JSON files, creating a scope per import because a DbContext is a scoped service.
Add Directory.Build.props with TreatWarningsAsErrors and analyzers, as shown in Analyzers, Code Style & Warnings.

Common mistakes

Putting parsing logic inside the worker class. It becomes hard to test; keep it in the library.
Catching every exception in the loop. Bugs then hide forever in the logs. Catch only what you can recover from.
Using Task.Delay with a fixed time after each import. The interval drifts as imports take longer; PeriodicTimer keeps a steady rhythm.
Forgetting CultureInfo.InvariantCulture when parsing numbers. A server set to another region can read 1.250 very differently.
Relative paths on a server. A Windows service's current folder is usually the system folder; configure absolute paths.

Key points

A worker service is a Generic Host app with one or more BackgroundService classes registered with AddHostedService.
Keep business logic in a class library that takes simple inputs, so it can be unit tested without the host.
PeriodicTimer plus the host's stoppingToken gives a clean polling loop that stops gracefully.
Validate options at start-up with ValidateDataAnnotations and ValidateOnStart.
Log with message templates, catch only recoverable errors, and write deterministic JSON output.
AddWindowsService and AddSystemd turn the same worker into an operating system service.

Exercise

Add a daily rejected lines report. Extend ImportSummary handling so that, in addition to the JSON summary, any file with at least one invalid line also produces <name>.errors.txt in the outbox, containing only the error lines, one per line. Write a test in ImportProcessorTests that imports a file with one bad line and checks the content of the .errors.txt file, and a second test proving that no .errors.txt is created when every line is valid.

Show solution

In ImportProcessor.ProcessFileAsync, after the summary JSON is written and before the CSV is moved:

if (parsed.Errors.Count > 0)
{
    string errorsPath = Path.Combine(outboxFolder, Path.ChangeExtension(fileName, ".errors.txt"));
    await File.WriteAllLinesAsync(errorsPath, parsed.Errors, ct);
}

Tests:

[Fact]
public async Task ProcessFile_WithBadLine_WritesErrorsFile()
{
    CancellationToken ct = TestContext.Current.CancellationToken;
    string csv = Path.Combine(_root, "shop-2.csv");
    await File.WriteAllLinesAsync(csv,
        ["OrderId,City,Item,Qty,UnitPrice", "C1,Pune,Pen,2,10", "C2,Pune,Ink,two,45"], ct);

    await ImportProcessor.ProcessFileAsync(csv, Path.Combine(_root, "out"), Path.Combine(_root, "arc"), ct);

    string[] lines = await File.ReadAllLinesAsync(Path.Combine(_root, "out", "shop-2.errors.txt"), ct);
    Assert.Equal("Line 3: Qty two is not a whole number", Assert.Single(lines));
}

[Fact]
public async Task ProcessFile_AllValid_WritesNoErrorsFile()
{
    CancellationToken ct = TestContext.Current.CancellationToken;
    string csv = Path.Combine(_root, "shop-3.csv");
    await File.WriteAllLinesAsync(csv, ["OrderId,City,Item,Qty,UnitPrice", "D1,Kochi,Bag,1,999"], ct);

    await ImportProcessor.ProcessFileAsync(csv, Path.Combine(_root, "out"), Path.Combine(_root, "arc"), ct);

    Assert.False(File.Exists(Path.Combine(_root, "out", "shop-3.errors.txt")));
}

The errors file is written only when there is something to report, which keeps the outbox clean. Both tests use the class's temporary root folder, so Dispose removes everything afterwards.

Quick Quiz

1. Which method of BackgroundService contains the long-running work?

The host calls ExecuteAsync once and passes a token that is cancelled at shutdown.

2. Why does the parser take IEnumerable<string> instead of a file path?

3. What does ValidateOnStart() add to options validation?

4. Why does the worker catch only IOException and UnauthorizedAccessException?