Mysql

80 posts in this section

Reading Flat Files: CSV, Fixed-Width, and Delimited with FlatFileItemReader

Introduction Flat files — CSV exports, fixed-width mainframe feeds, pipe-delimited data dumps — are the most common input source for batch jobs. Spring Batch’s FlatFileItemReader handles all of them. It is restartable out of the box: it persists its line-number position in the ExecutionContext so that a restarted job resumes exactly where it crashed. In this article you will build a complete order-import job that reads a CSV file and inserts rows into a MySQL orders table.

Continue reading »

Reading from External Sources: REST APIs, S3, and Custom ItemReaders

Introduction Not all batch input comes from files or databases. You may need to pull orders from an e-commerce API, sync products from a supplier feed, or process CSV exports stored in Amazon S3. Spring Batch provides MultiResourceItemReader for S3 files and a clean ItemReader interface for anything else. In this article you will build: A custom ItemReader that pages through a REST API An S3 reader that downloads files on demand A composite reader that merges multiple sources into one step The ItemReader Contract The entire ItemReader interface is one method:

Continue reading »

Reading from MySQL: JdbcCursorItemReader and JdbcPagingItemReader

Introduction Most real batch jobs read from a database, not a file. Spring Batch provides two JDBC readers for this: Reader Strategy Thread-safe Use when JdbcCursorItemReader Open a server-side cursor, stream rows No Single-threaded step, huge result sets JdbcPagingItemReader Execute LIMIT / OFFSET queries in a loop Yes Multi-threaded steps, sorted data Both handle restartability automatically and both work with any DataSource — including MySQL. JdbcCursorItemReader How it works The reader opens a single JDBC ResultSet on Step.

Continue reading »

Reading with JPA: JpaPagingItemReader and Entity-Based Reading

Introduction When your application already uses JPA/Hibernate, JpaPagingItemReader lets you read data using JPQL queries and mapped entities instead of raw JDBC. You get the full object graph, type-safe queries, and familiar entity lifecycle — but you also inherit JPA’s pitfalls: the N+1 problem, session-per-read overhead, and first-level cache growth. This article covers: When to choose JpaPagingItemReader over JdbcPagingItemReader Setting up the reader with JPQL and named queries Fetching associations to avoid N+1 Clearing the persistence context to prevent memory leaks A complete order-processing example with MySQL When to Use JpaPagingItemReader Use it when:

Continue reading »

Retry Logic: Handling Transient Failures Gracefully

Introduction Batch jobs interact with databases, REST APIs, and file systems — all of which fail transiently. A MySQL deadlock resolves itself in milliseconds. A network timeout to an external service clears up in seconds. Retrying these transient failures automatically is far better than failing the entire job and requiring a manual restart. Spring Batch has built-in retry support at the step level, integrated with its transaction management. This article covers everything you need to configure robust retry behaviour.

Continue reading »

Rollback in Production: Tag-Based Strategies and CI Validation

Article 9 covered what rollback commands exist. This article covers how to make rollback reliable in production — where a slow or broken rollback costs real money and real users. The difference between knowing the commands and having a working rollback strategy is process: mandatory tagging, pre-generated rollback files, CI gates, and a decision tree that removes guesswork during an incident. Why Rollback Fails in Production Rollback failures in production share a small set of root causes:

Continue reading »

Rollback Strategies: Automatic, Custom, Tag-Based, and Count-Based

Rollback is the thing you build at 2am when the deployment broke production. The problem is that 2am is exactly the wrong time to discover your rollback blocks are missing, incomplete, or untested. This article covers rollback from a strategy perspective: how Liquibase generates rollback, when you must write it yourself, how to handle change types that cannot be rolled back at all, and how to validate your rollback strategy in CI so it works when you need it.

Continue reading »

Skip Logic, Dead Letter Patterns, and Job Restart Strategies

Introduction Retry handles transient failures. Skip handles permanent ones — bad data rows, constraint violations, malformed records that will never succeed no matter how many times you retry. Skip logic lets your job continue processing good records while recording bad ones for human review. This article covers: Configuring skip for specific exception types Custom SkipPolicy for fine-grained control Dead-letter table pattern for tracking skipped items Stopping a job intentionally vs failing it Handling abandoned executions Designing jobs that restart safely after any failure Basic Skip Configuration return new StepBuilder("importOrdersStep", jobRepository) .

Continue reading »

Spring Boot + Spring Batch Setup: Your First Complete Batch Project

Article 2 showed how chunk processing works conceptually. This article sets up the real project you’ll build on for the rest of the series — complete Maven configuration, MySQL metadata tables, multiple ways to trigger jobs, and correct use of JobParameters. Everything in this article is production-ready from the start. Project Dependencies Add spring-boot-starter-batch to your Maven project. That one starter brings in everything you need. <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.

Continue reading »

Spring Boot Integration: Zero-Config Setup and Full Properties Reference

Running Liquibase from the CLI is fine for learning and for standalone scripts. In a real Spring Boot application, you want migrations to run automatically at startup — before the application code touches the database. Spring Boot’s Liquibase auto-configuration handles this with zero boilerplate: add the dependency, point to your changelog, and migrations run before the application context is ready. This article covers the complete integration: Maven/Gradle setup, how auto-run works and why, the full spring.

Continue reading »