Содержание
- 13.1. Setting the Storage Engine
- 13.2. Overview of MySQL Storage Engine Architecture
- 13.3. The
InnoDB
Storage Engine - 13.3.1. InnoDB as the Default MySQL Storage Engine
- 13.3.2. Configuring
InnoDB
- 13.3.3. Using Per-Table Tablespaces
- 13.3.4.
InnoDB
Startup Options and System Variables - 13.3.5. Creating and Using
InnoDB
Tables - 13.3.6. Adding, Removing, or Resizing
InnoDB
Data and Log Files - 13.3.7. Backing Up and Recovering an
InnoDB
Database - 13.3.8. Moving an
InnoDB
Database to Another Machine - 13.3.9. The
InnoDB
Transaction Model and Locking - 13.3.10.
InnoDB
Multi-Versioning - 13.3.11.
InnoDB
Table and Index Structures - 13.3.12.
InnoDB
Disk I/O and File Space Management - 13.3.13.
InnoDB
Error Handling - 13.3.14.
InnoDB
Performance Tuning and Troubleshooting - 13.3.15. Limits on
InnoDB
Tables
- 13.4. New Features of InnoDB 1.1
- 13.4.1. Introduction to InnoDB 1.1
- 13.4.2. Fast Index Creation in the InnoDB Storage Engine
- 13.4.3. InnoDB Data Compression
- 13.4.4. InnoDB File-Format Management
- 13.4.5. How InnoDB Stores Variable-Length Columns
- 13.4.6. InnoDB
INFORMATION_SCHEMA
tables - 13.4.7. InnoDB Performance and Scalability Enhancements
- 13.4.8. Changes for Flexibility, Ease of Use and Reliability
- 13.4.9. Installing the InnoDB Storage Engine
- 13.4.10. Upgrading the InnoDB Storage Engine
- 13.4.11. Downgrading the InnoDB Storage Engine
- 13.4.12. InnoDB Storage Engine Change History
- 13.4.13. Third-Party Software
- 13.4.14. List of Parameters Changed in InnoDB 1.1 and InnoDB Plugin 1.0
- 13.5. The
MyISAM
Storage Engine - 13.6. The
MEMORY
Storage Engine - 13.7. The
CSV
Storage Engine - 13.8. The
ARCHIVE
Storage Engine - 13.9. The
BLACKHOLE
Storage Engine - 13.10. The
MERGE
Storage Engine - 13.11. The
FEDERATED
Storage Engine - 13.12. The
EXAMPLE
Storage Engine - 13.13. Other Storage Engines
MySQL supports several storage engines that act as handlers for different table types. MySQL storage engines include both those that handle transaction-safe tables and those that handle nontransaction-safe tables.
MySQL Server uses a pluggable storage engine architecture that enables storage engines to be loaded into and unloaded from a running MySQL server.
To determine which storage engines your server supports by using the
SHOW ENGINES
statement. The value in
the Support
column indicates whether an engine
can be used. A value of YES
,
NO
, or DEFAULT
indicates that
an engine is available, not available, or available and currently
set as the default storage engine.
mysql> SHOW ENGINES\G
*************************** 1. row ***************************
Engine: FEDERATED
Support: NO
Comment: Federated MySQL storage engine
Transactions: NULL
XA: NULL
Savepoints: NULL
*************************** 2. row ***************************
Engine: MRG_MYISAM
Support: YES
Comment: Collection of identical MyISAM tables
Transactions: NO
XA: NO
Savepoints: NO
*************************** 3. row ***************************
Engine: MyISAM
Support: DEFAULT
Comment: Default engine as of MySQL 3.23 with great performance
Transactions: NO
XA: NO
Savepoints: NO
...
This chapter describes each of the MySQL storage engines except for
NDBCLUSTER
, which is covered in
MySQL Cluster NDB 6.X/7.X. It also contains a
description of the pluggable storage engine architecture (see
Section 13.2, “Overview of MySQL Storage Engine Architecture”).
For information about storage engine support offered in commercial MySQL Server binaries, see MySQL Enterprise Server 5.1, on the MySQL Web site. The storage engines available might depend on which edition of Enterprise Server you are using.
For answers to some commonly asked questions about MySQL storage engines, see Section B.2, “MySQL 5.5 FAQ: Storage Engines”.
MySQL 5.5 supported storage engines
InnoDB
: A transaction-safe (ACID compliant) storage engine for MySQL that has commit, rollback, and crash-recovery capabilities to protect user data.InnoDB
row-level locking (without escalation to coarser granularity locks) and Oracle-style consistent nonlocking reads increase multi-user concurrency and performance.InnoDB
stores user data in clustered indexes to reduce I/O for common queries based on primary keys. To maintain data integrity,InnoDB
also supportsFOREIGN KEY
referential-integrity constraints.InnoDB
is the default storage engine as of MySQL 5.5.5.MyISAM
: The MySQL storage engine that is used the most in Web, data warehousing, and other application environments.MyISAM
is supported in all MySQL configurations, and is the default storage engine prior to MySQL 5.5.5.Memory
: Stores all data in RAM for extremely fast access in environments that require quick lookups of reference and other like data. This engine was formerly known as theHEAP
engine.Merge
: Enables a MySQL DBA or developer to logically group a series of identicalMyISAM
tables and reference them as one object. Good for VLDB environments such as data warehousing.Archive
: Provides the perfect solution for storing and retrieving large amounts of seldom-referenced historical, archived, or security audit information.Federated
: Offers the ability to link separate MySQL servers to create one logical database from many physical servers. Very good for distributed or data mart environments.CSV
: The CSV storage engine stores data in text files using comma-separated values format. You can use the CSV engine to easily exchange data between other software and applications that can import and export in CSV format.Blackhole
: The Blackhole storage engine accepts but does not store data and retrievals always return an empty set. The functionality can be used in distributed database design where data is automatically replicated, but not stored locally.Пример
: The Пример storage engine is “stub” engine that does nothing. You can create tables with this engine, but no data can be stored in them or retrieved from them. The purpose of this engine is to serve as an example in the MySQL source code that illustrates how to begin writing new storage engines. As such, it is primarily of interest to developers.
It is important to remember that you are not restricted to using the same storage engine for an entire server or schema: you can use a different storage engine for each table in your schema.
Choosing a Storage Engine
The various storage engines provided with MySQL are designed with different use cases in mind. To use the pluggable storage architecture effectively, it is good to have an idea of the advantages and disadvantages of the various storage engines. The following table provides an overview of some storage engines provided with MySQL:
Table 13.1. Storage Engines Feature Summary
Feature | MyISAM | Memory | InnoDB | Archive | NDB |
---|---|---|---|---|---|
Storage limits | 256TB | RAM | 64TB | None | 384EB |
Transactions | No | No | Yes | No | Yes |
Locking granularity | Table | Table | Row | Table | Row |
MVCC | No | No | Yes | No | No |
Geospatial data type support | Yes | No | Yes | Yes | Yes |
Geospatial indexing support | Yes | No | No | No | No |
B-tree indexes | Yes | Yes | Yes | No | Yes |
Hash indexes | No | Yes | No[a] | No | Yes |
Full-text search indexes | Yes | No | No | No | No |
Clustered indexes | No | No | Yes | No | No |
Data caches | No | N/A | Yes | No | Yes |
Index caches | Yes | N/A | Yes | No | Yes |
Compressed data | Yes[b] | No | Yes[c] | Yes | No |
Encrypted data[d] | Yes | Yes | Yes | Yes | Yes |
Cluster database support | No | No | No | No | Yes |
Replication support[e] | Yes | Yes | Yes | Yes | Yes |
Foreign key support | No | No | Yes | No | No |
Backup / point-in-time recovery[f] | Yes | Yes | Yes | Yes | Yes |
Query cache support | Yes | Yes | Yes | Yes | Yes |
Update statistics for data dictionary | Yes | Yes | Yes | Yes | Yes |
[a] InnoDB utilizes hash indexes internally for its Adaptive Hash Index feature. [b] Compressed MyISAM tables are supported only when using the compressed row format. Tables using the compressed row format with MyISAM are read only. [c] Compressed InnoDB tables require the InnoDB Barracuda file format. [d] Implemented in the server (via encryption functions), rather than in the storage engine. [e] Implemented in the server, rather than in the storage product. [f] Implemented in the server, rather than in the storage product. |
When you create a new table, you can specify which storage engine
to use by adding an ENGINE
table option to the
CREATE TABLE
statement:
CREATE TABLE t (i INT) ENGINE = INNODB;
If you omit the ENGINE
option, the default
storage engine is used. The default engine is
InnoDB
as of MySQL 5.5.5
(MyISAM
before 5.5.5). You can
specify the default engine by using the
--default-storage-engine
server
startup option, or by setting the
default-storage-engine
option in
the my.cnf
configuration file.
You can set the default storage engine to be used during the
current session by setting the
default_storage_engine
variable:
SET default_storage_engine=MYISAM;
When MySQL is installed on Windows using the MySQL Configuration
Wizard, the InnoDB
or MyISAM
storage engine can be selected as the default. See
Section 2.3.5.5, “The Database Usage Dialog”.
To convert a table from one storage engine to another, use an
ALTER TABLE
statement that
indicates the new engine:
ALTER TABLE t ENGINE = MYISAM;
See Section 12.1.17, “CREATE TABLE
Синтаксис”, and
Section 12.1.7, “ALTER TABLE
Синтаксис”.
If you try to use a storage engine that is not compiled in or that is compiled in but deactivated, MySQL instead creates a table using the default storage engine. This behavior is convenient when you want to copy tables between MySQL servers that support different storage engines. (For example, in a replication setup, perhaps your master server supports transactional storage engines for increased safety, but the slave servers use only nontransactional storage engines for greater speed.)
This automatic substitution of the default storage engine for
unavailable engines can be confusing for new MySQL users. A
warning is generated whenever a storage engine is automatically
changed. To prevent this from happening if the desired engine is
unavailable, enable the
NO_ENGINE_SUBSTITUTION
SQL mode.
In this case, an error occurs instead of a warning and the table
is not created or altered if the desired engine is unavailable.
See Section 5.1.6, “Server SQL Modes”.
For new tables, MySQL always creates an .frm
file to hold the table and column definitions. The table's index
and data may be stored in one or more other files, depending on
the storage engine. The server creates the
.frm
file above the storage engine level.
Individual storage engines create any additional files required
for the tables that they manage. If a table name contains special
characters, the names for the table files contain encoded versions
of those characters as described in
Section 8.2.3, “Mapping of Identifiers to File Names”.
A database may contain tables of different types. That is, tables need not all be created with the same storage engine.
The MySQL pluggable storage engine architecture enables a database professional to select a specialized storage engine for a particular application need while being completely shielded from the need to manage any specific application coding requirements. The MySQL server architecture isolates the application programmer and DBA from all of the low-level implementation details at the storage level, providing a consistent and easy application model and API. Thus, although there are different capabilities across different storage engines, the application is shielded from these differences.
The MySQL pluggable storage engine architecture is shown in Figure 13.1, “MySQL Architecture with Pluggable Storage Engines”.
The pluggable storage engine architecture provides a standard set of management and support services that are common among all underlying storage engines. The storage engines themselves are the components of the database server that actually perform actions on the underlying data that is maintained at the physical server level.
This efficient and modular architecture provides huge benefits for those wishing to specifically target a particular application need—such as data warehousing, transaction processing, or high availability situations—while enjoying the advantage of utilizing a set of interfaces and services that are independent of any one storage engine.
The application programmer and DBA interact with the MySQL database through Connector APIs and service layers that are above the storage engines. If application changes bring about requirements that demand the underlying storage engine change, or that one or more storage engines be added to support new needs, no significant coding or process changes are required to make things work. The MySQL server architecture shields the application from the underlying complexity of the storage engine by presenting a consistent and easy-to-use API that applies across storage engines.
MySQL Server uses a pluggable storage engine architecture that enables storage engines to be loaded into and unloaded from a running MySQL server.
Plugging in a Storage Engine
Before a storage engine can be used, the storage engine plugin
shared library must be loaded into MySQL using the
INSTALL PLUGIN
statement. For
example, if the EXAMPLE
engine plugin is
named example
and the shared library is named
ha_example.so
, you load it with the
following statement:
mysql> INSTALL PLUGIN example SONAME 'ha_example.so';
To install a pluggable storage engine, the plugin file must be
located in the MySQL plugin directory, and the user issuing the
INSTALL PLUGIN
statement must
have INSERT
privilege for the
mysql.plugin
table.
The shared library must be located in the MySQL server plugin
directory, the location of which is given by the
plugin_dir
system variable.
Unplugging a Storage Engine
To unplug a storage engine, use the
UNINSTALL PLUGIN
statement:
mysql> UNINSTALL PLUGIN example;
If you unplug a storage engine that is needed by existing tables, those tables become inaccessible, but will still be present on disk (where applicable). Ensure that there are no tables using a storage engine before you unplug the storage engine.
A MySQL pluggable storage engine is the component in the MySQL database server that is responsible for performing the actual data I/O operations for a database as well as enabling and enforcing certain feature sets that target a specific application need. A major benefit of using specific storage engines is that you are only delivered the features needed for a particular application, and therefore you have less system overhead in the database, with the end result being more efficient and higher database performance. This is one of the reasons that MySQL has always been known to have such high performance, matching or beating proprietary monolithic databases in industry standard benchmarks.
From a technical perspective, what are some of the unique supporting infrastructure components that are in a storage engine? Some of the key feature differentiations include:
Concurrency: Some applications have more granular lock requirements (such as row-level locks) than others. Choosing the right locking strategy can reduce overhead and therefore improve overall performance. This area also includes support for capabilities such as multi-version concurrency control or “snapshot” read.
Transaction Support: Not every application needs transactions, but for those that do, there are very well defined requirements such as ACID compliance and more.
Referential Integrity: The need to have the server enforce relational database referential integrity through DDL defined foreign keys.
Physical Storage: This involves everything from the overall page size for tables and indexes as well as the format used for storing data to physical disk.
Index Support: Different application scenarios tend to benefit from different index strategies. Each storage engine generally has its own indexing methods, although some (such as B-tree indexes) are common to nearly all engines.
Memory Caches: Different applications respond better to some memory caching strategies than others, so although some memory caches are common to all storage engines (such as those used for user connections or MySQL's high-speed Query Cache), others are uniquely defined only when a particular storage engine is put in play.
Performance Aids: This includes multiple I/O threads for parallel operations, thread concurrency, database checkpointing, bulk insert handling, and more.
Miscellaneous Target Features: This may include support for geospatial operations, security restrictions for certain data manipulation operations, and other similar features.
Each set of the pluggable storage engine infrastructure components are designed to offer a selective set of benefits for a particular application. Conversely, avoiding a set of component features helps reduce unnecessary overhead. It stands to reason that understanding a particular application's set of requirements and selecting the proper MySQL storage engine can have a dramatic impact on overall system efficiency and performance.
- 13.3.1. InnoDB as the Default MySQL Storage Engine
- 13.3.2. Configuring
InnoDB
- 13.3.3. Using Per-Table Tablespaces
- 13.3.4.
InnoDB
Startup Options and System Variables - 13.3.5. Creating and Using
InnoDB
Tables - 13.3.6. Adding, Removing, or Resizing
InnoDB
Data and Log Files - 13.3.7. Backing Up and Recovering an
InnoDB
Database - 13.3.8. Moving an
InnoDB
Database to Another Machine - 13.3.9. The
InnoDB
Transaction Model and Locking - 13.3.10.
InnoDB
Multi-Versioning - 13.3.11.
InnoDB
Table and Index Structures - 13.3.12.
InnoDB
Disk I/O and File Space Management - 13.3.13.
InnoDB
Error Handling - 13.3.14.
InnoDB
Performance Tuning and Troubleshooting - 13.3.15. Limits on
InnoDB
Tables
InnoDB
is a high-reliability and high-performance
storage engine for MySQL. Starting with MySQL 5.5, it is the default
MySQL storage engine. Key advantages of InnoDB include:
Its design follows the ACID model, with transactions featuring commit, rollback, and crash-recovery capabilities to protect user data.
Row-level locking and Oracle-style consistent reads increase multi-user concurrency and performance.
InnoDB
tables arrange your data on disk to optimize common queries based on primary keys. EachInnoDB
table has a primary key index called the clustered index that organizes the data to minimize I/O for primary key lookups.To maintain data integrity,
InnoDB
also supportsFOREIGN KEY
referential-integrity constraints.You can freely mix
InnoDB
tables with tables from other MySQL storage engines, even within the same statement. For example, you can use a join operation to combine data fromInnoDB
andMEMORY
tables in a single query.
To determine whether your server supports InnoDB
use the SHOW ENGINES
statement. See
Section 12.7.5.17, “SHOW ENGINES
Синтаксис”.
Table 13.2. InnoDB
Storage Engine
Features
Storage limits | 64TB | Transactions | Yes | Locking granularity | Row |
MVCC | Yes | Geospatial data type support | Yes | Geospatial indexing support | No |
B-tree indexes | Yes | Hash indexes | No[a] | Full-text search indexes | No |
Clustered indexes | Yes | Data caches | Yes | Index caches | Yes |
Compressed data | Yes[b] | Encrypted data[c] | Yes | Cluster database support | No |
Replication support[d] | Yes | Foreign key support | Yes | Backup / point-in-time recovery[e] | Yes |
Query cache support | Yes | Update statistics for data dictionary | Yes | ||
[a] InnoDB utilizes hash indexes internally for its Adaptive Hash Index feature. [b] Compressed InnoDB tables require the InnoDB Barracuda file format. [c] Implemented in the server (via encryption functions), rather than in the storage engine. [d] Implemented in the server, rather than in the storage product. [e] Implemented in the server, rather than in the storage product. |
InnoDB
has been designed for maximum performance
when processing large data volumes. Its CPU efficiency is probably
not matched by any other disk-based relational database engine.
The InnoDB
storage engine maintains its own
buffer pool for caching data and indexes in main memory.
InnoDB
stores its tables and indexes in a
tablespace, which may consist of several files (or raw disk
partitions). This is different from, for example,
MyISAM
tables where each table is stored using
separate files. InnoDB
tables can be very large
even on operating systems where file size is limited to 2GB.
InnoDB
is published under the same GNU GPL
License Version 2 (of June 1991) as MySQL. For more information on
MySQL licensing, see http://www.mysql.com/company/legal/licensing/.
Additional Resources
A forum dedicated to the
InnoDB
storage engine is available at http://forums.mysql.com/list.php?22.The InnoDB storage engine in MySQL 5.5 releases includes a number performance improvements that in MySQL 5.1 were only available by installing the InnoDB Plugin. This latest InnoDB (now known as InnoDB 1.1) offers new features, improved performance and scalability, enhanced reliability and new capabilities for flexibility and ease of use. Among the top features are Fast Index Creation, table and index compression, file format management, new
INFORMATION_SCHEMA
tables, capacity tuning, multiple background I/O threads, multiple buffer pools, and group commit.For information about these features, see Section 13.4, “New Features of InnoDB 1.1”.
The MySQL Enterprise Backup product lets you back up a running MySQL database, including
InnoDB
andMyISAM
tables, with minimal disruption to operations while producing a consistent snapshot of the database. When MySQL Enterprise Backup is copyingInnoDB
tables, reads and writes to bothInnoDB
andMyISAM
tables can continue. During the copying ofMyISAM
and other non-InnoDB tables, reads (but not writes) to those tables are permitted. In addition, MySQL Enterprise Backup can create compressed backup files, and back up subsets ofInnoDB
tables. In conjunction with the MySQL binary log, you can perform point-in-time recovery. MySQL Enterprise Backup is included as part of the MySQL Enterprise subscription.For a more complete description of MySQL Enterprise Backup, see MySQL Enterprise Backup User's Guide (Version 3.5.4).
MySQL has a well-earned reputation for being easy-to-use and delivering performance and scalability. In previous versions of MySQL, MyISAM was the default storage engine. In our experience, most users never changed the default settings. With MySQL 5.5, InnoDB becomes the default storage engine. Again, we expect most users will not change the default settings. But, because of InnoDB, the default settings deliver the benefits users expect from their RDBMS: ACID Transactions, Referential Integrity, and Crash Recovery. Let's explore how using InnoDB tables improves your life as a MySQL user, DBA, or developer.
Trends in Storage Engine Usage
In the first years of MySQL growth, early web-based applications didn't push the limits of concurrency and availability. In 2010, hard drive and memory capacity and the performance/price ratio have all gone through the roof. Users pushing the performance boundaries of MySQL care a lot about reliability and crash recovery. MySQL databases are big, busy, robust, distributed, and important.
InnoDB hits the sweet spot of these top user priorities. The trend of storage engine usage has shifted in favor of the more efficient InnoDB. With MySQL 5.5, the time is right to make InnoDB the default storage engine.
Consequences of InnoDB as Default MySQL Storage Engine
Starting from MySQL 5.5.5, the default storage engine for new
tables is InnoDB. This change applies to newly created tables that
don't specify a storage engine with a clause such as
ENGINE=MyISAM
. (Given this change of default
behavior, MySQL 5.5 might be a logical point to evaluate whether
your tables that do use MyISAM could benefit from switching to
InnoDB.)
The mysql
and
information_schema
databases, that implement
some of the MySQL internals, still use MyISAM. In particular, you
cannot switch the grant tables to use InnoDB.
Benefits of InnoDB Tables
If you use MyISAM tables but aren't tied to them for technical reasons, you'll find many things more convenient when you use InnoDB tables in MySQL 5.5:
If your server crashes because of a hardware or software issue, regardless of what was happening in the database at the time, you don't need to do anything special after restarting the database. InnoDB automatically finalizes any changes that were committed before the time of the crash, and undoes any changes that were in process but not committed. Just restart and continue where you left off.
The InnoDB buffer pool caches table and index data as the data is accessed. Frequently used data is processed directly from memory. This cache applies to so many types of information, and speeds up processing so much, that dedicated database servers assign up to 80% of their physical memory to the InnoDB buffer pool.
If you split up related data into different tables, you can set up foreign keys that enforce referential integrity. Update or delete data, and the related data in other tables is updated or deleted automatically. Try to insert data into a secondary table without corresponding data in the primary table, and it gets kicked out automatically.
If data becomes corrupted on disk or in memory, a checksum mechanism alerts you to the bogus data before you use it.
When you design your database with appropriate primary key columns for each table, operations involving those columns are automatically optimized. It is very fast to reference the primary key columns in
WHERE
clauses,ORDER BY
clauses,GROUP BY
clauses, and join operations.Inserts, updates, deletes are optimized by an automatic mechanism called change buffering. InnoDB not only allows concurrent read and write access to the same table, it caches changed data to streamline disk I/O.
Performance benefits are not limited to giant tables with long-running queries. When the same rows are accessed over and over from a table, a feature called the Adaptive Hash Index takes over to make these lookups even faster, as if they came out of a hash table.
Best Practices for InnoDB Tables
If you have been using InnoDB for a long time, you already know about features like transactions and foreign keys. If not, read about them throughout this chapter. To make a long story short:
Specify a primary key for every table using the most frequently queried column or columns, or an auto-increment value if there isn't an obvious primary key.
Embrace the idea of joins, where data is pulled from multiple tables based on identical ID values from those tables. For fast join performance, define foreign keys on the join columns, and declare those columns with the same datatype in each table. The foreign keys also propagate deletes or updates to all affected tables, and prevent insertion of data in a child table if the corresponding IDs are not present in the parent table.
Turn off autocommit. Committing hundreds of times a second puts a cap on performance (limited by the write speed of your storage device).
Bracket sets of related changes, logical units of work, with
START TRANSACTION
andCOMMIT
statements. While you don't want to commit too often, you also don't want to issue huge batches ofINSERT
,UPDATE
, orDELETE
statements that run for hours without committing.Stop using
LOCK TABLE
statements. InnoDB can handle multiple sessions all reading and writing to the same table at once, without sacrificing reliability or high performance. To get exclusive write access to a set of rows, use theSELECT ... FOR UPDATE
syntax to lock just the rows you intend to update.Enable the
innodb_file_per_table
option to put the data and indexes for individual tables into separate files, instead of in a single giant system tablespace. (This setting is required to use some of the other features, such as table compression and fast truncation.)Evaluate whether your data and access patterns benefit from the new InnoDB table compression feature (
ROW_FORMAT=COMPRESSED
on theCREATE TABLE
statement. You can compress InnoDB tables without sacrificing read/write capability.Run your server with the option
--sql_mode=NO_ENGINE_SUBSTITUTION
to prevent tables being created with a different storage engine if there is an issue with the one specified in theENGINE=
clause ofCREATE TABLE
.
Recent Improvements for InnoDB Tables (from the Plugin Era)
If you have experience with InnoDB, but not the recent incarnation known as the InnoDB Plugin, read about the latest enhancements in Section 13.4, “New Features of InnoDB 1.1”. To make a long story short:
You can compress tables and associated indexes.
You can create and drop indexes with much less performance or availability impact than before.
Truncating a table is very fast, and can free up disk space for the operating system to reuse, rather than freeing up space within the system tablespace that only InnoDB could reuse.
The storage layout for table data is more efficient for BLOBs and long text fields.
You can monitor the internal workings of the storage engine by querying
INFORMATION_SCHEMA
tables.You can monitor the performance details of the storage engine by querying
performance_schema
tables.There are many many performance improvements. In particular, crash recovery, the automatic process that makes all data consistent when the database is restarted, is fast and reliable. (Now much much faster than long-time InnoDB users are used to.) The bigger the database, the more dramatic the speedup.
Most new performance features are automatic, or at most require setting a value for a configuration option. For details, see Section 13.4.7, “InnoDB Performance and Scalability Enhancements”. For InnoDB-specific tuning techniques you can apply in your application code, see Section 7.5, “Optimizing for
InnoDB
Tables”. Advanced users can review Section 13.3.4, “InnoDB
Startup Options and System Variables”.
Testing and Benchmarking with InnoDB as Default Storage Engine
Even before completing your upgrade to MySQL 5.5, you can preview
whether your database server or application works correctly with
InnoDB as the default storage engine. To set up InnoDB as the
default storage engine with an earlier MySQL release, either
specify on the command line
--default-storage-engine=InnoDB
, or add to your
my.cnf
file
default-storage-engine=innodb
in the
[mysqld]
section, then restart the server.
Since changing the default storage engine only affects new tables
as they are created, run all your application installation and
setup steps to confirm that everything installs properly. Then
exercise all the application features to make sure all the data
loading, editing, and querying features work. If a table relies on
some MyISAM-specific feature, you'll receive an error; add the
ENGINE=MyISAM
clause to the CREATE
TABLE
statement to avoid the error. (For example, tables
that rely on full-text search must be MyISAM tables rather than
InnoDB ones.)
If you didn't make a deliberate decision about the storage engine,
and you just want to preview how certain tables work when they're
created under InnoDB, issue the command ALTER TABLE
table_name ENGINE=InnoDB;
for each table. Or, to run
test queries and other statements without disturbing the original
table, make a copy like so:
CREATE TABLE InnoDB_Table (...) ENGINE=InnoDB AS SELECT *
FROM MyISAM_Table;
Since there are so many performance enhancements in the InnoDB that is part of MySQL 5.5, to get a true idea of the performance with a full application under a realistic workload, install the real MySQL 5.5 and run benchmarks.
Test the full application lifecycle, from installation, through heavy usage, and server restart. Kill the server process while the database is busy to simulate a power failure, and verify that the data is recovered successfully when you restart the server.
Test any replication configurations, especially if you use different MySQL versions and options on the master and the slaves.
Verifying that InnoDB is the Default Storage Engine
To know what the status of InnoDB is, whether you're doing what-if testing with an older MySQL or comprehensive testing with MySQL 5.5:
Issue the command
SHOW VARIABLES LIKE 'have_innodb';
to confirm that InnoDB is available at all. If the result isNO
, you have a mysqld binary that was compiled without InnoDB support and you need to get a different one. If the result isDISABLED
, go back through your startup options and configuration file and get rid of anyskip-innodb
option.Issue the command
SHOW ENGINES;
to see all the different MySQL storage engines. Look forDEFAULT
in the InnoDB line.
The first decisions to make about InnoDB configuration involve how to lay out InnoDB data files, and how much memory to allocate for the InnoDB storage engine. You record these choices either by recording them in a configuration file that MySQL reads at startup, or by specifying them as command-line options in a startup script.
Overview of InnoDB Tablespace and Log Files
Two important disk-based resources managed by the
InnoDB
storage engine are its tablespace data
files and its log files. If you specify no
InnoDB
configuration options, MySQL creates an
auto-extending 10MB data file named ibdata1
and two 5MB log files named ib_logfile0
and
ib_logfile1
in the MySQL data directory. To
get good performance, explicitly provide InnoDB
parameters as discussed in the following examples. Naturally, edit
the settings to suit your hardware and requirements.
The examples shown here are representative. See
Section 13.3.4, “InnoDB
Startup Options and System Variables” for additional information
about InnoDB
-related configuration parameters.
Considerations for Storage Devices
In some cases, database performance improves if the data is not
all placed on the same physical disk. Putting log files on a
different disk from data is very often beneficial for performance.
The example illustrates how to do this. It places the two data
files on different disks and places the log files on the third
disk. InnoDB
fills the tablespace beginning
with the first data file. You can also use raw disk partitions
(raw devices) as InnoDB
data files, which may
speed up I/O. See Section 13.3.3.1, “Using Raw Devices for the Shared Tablespace”.
InnoDB
is a transaction-safe (ACID compliant)
storage engine for MySQL that has commit, rollback, and
crash-recovery capabilities to protect user data.
However, it cannot do so if the
underlying operating system or hardware does not work as
advertised. Many operating systems or disk subsystems may delay
or reorder write operations to improve performance. On some
operating systems, the very fsync()
system
call that should wait until all unwritten data for a file has
been flushed might actually return before the data has been
flushed to stable storage. Because of this, an operating system
crash or a power outage may destroy recently committed data, or
in the worst case, even corrupt the database because of write
operations having been reordered. If data integrity is important
to you, perform some “pull-the-plug” tests before
using anything in production. On Mac OS X 10.3 and up,
InnoDB
uses a special
fcntl()
file flush method. Under Linux, it is
advisable to disable the write-back
cache.
On ATA/SATA disk drives, a command such hdparm -W0
/dev/hda
may work to disable the write-back cache.
Beware that some drives or disk
controllers may be unable to disable the write-back
cache.
If reliability is a consideration for your data, do not
configure InnoDB
to use data files or log
files on NFS volumes. Potential problems vary according to OS
and version of NFS, and include such issues as lack of
protection from conflicting writes, and limitations on maximum
file sizes.
Specifying the Location and Size for InnoDB Tablespace Files
To set up the InnoDB
tablespace files, use the
innodb_data_file_path
option in
the [mysqld]
section of the
my.cnf
option file. On Windows, you can use
my.ini
instead. The value of
innodb_data_file_path
should be a
list of one or more data file specifications. If you name more
than one data file, separate them by semicolon
(“;
”) characters:
innodb_data_file_path=datafile_spec1
[;datafile_spec2
]...
For example, the following setting explicitly creates a tablespace having the same characteristics as the default:
[mysqld] innodb_data_file_path=ibdata1:10M:autoextend
This setting configures a single 10MB data file named
ibdata1
that is auto-extending. No location
for the file is given, so by default, InnoDB
creates it in the MySQL data directory.
Sizes are specified using K
,
M
, or G
suffix letters to
indicate units of KB, MB, or GB.
A tablespace containing a fixed-size 50MB data file named
ibdata1
and a 50MB auto-extending file named
ibdata2
in the data directory can be
configured like this:
[mysqld] innodb_data_file_path=ibdata1:50M;ibdata2:50M:autoextend
The full syntax for a data file specification includes the file name, its size, and several optional attributes:
file_name
:file_size
[:autoextend[:max:max_file_size
]]
The autoextend
and max
attributes can be used only for the last data file in the
innodb_data_file_path
line.
If you specify the autoextend
option for the
last data file, InnoDB
extends the data file if
it runs out of free space in the tablespace. The increment is 8MB
at a time by default. To modify the increment, change the
innodb_autoextend_increment
system variable.
If the disk becomes full, you might want to add another data file
on another disk. For tablespace reconfiguration instructions, see
Section 13.3.6, “Adding, Removing, or Resizing InnoDB
Data and Log
Files”.
InnoDB
is not aware of the file system maximum
file size, so be cautious on file systems where the maximum file
size is a small value such as 2GB. To specify a maximum size for
an auto-extending data file, use the max
attribute following the autoextend
attribute.
Use the max
attribute only in cases where
constraining disk usage is of critical importance, because
exceeding the maximum size causes a fatal error, possibly
including a crash. The following configuration permits
ibdata1
to grow up to a limit of 500MB:
[mysqld] innodb_data_file_path=ibdata1:10M:autoextend:max:500M
InnoDB
creates tablespace files in the MySQL
data directory by default. To specify a location explicitly, use
the innodb_data_home_dir
option.
For example, to use two files named ibdata1
and ibdata2
but create them in the
/ibdata
directory, configure
InnoDB
like this:
[mysqld] innodb_data_home_dir = /ibdata innodb_data_file_path=ibdata1:50M;ibdata2:50M:autoextend
InnoDB
does not create directories, so make
sure that the /ibdata
directory exists
before you start the server. This is also true of any log file
directories that you configure. Use the Unix or DOS
mkdir
command to create any necessary
directories.
Make sure that the MySQL server has the proper access rights to create files in the data directory. More generally, the server must have access rights in any directory where it needs to create data files or log files.
InnoDB
forms the directory path for each data
file by textually concatenating the value of
innodb_data_home_dir
to the data
file name, adding a path name separator (slash or backslash)
between values if necessary. If the
innodb_data_home_dir
option is
not specified in my.cnf
at all, the default
value is the “dot” directory ./
,
which means the MySQL data directory. (The MySQL server changes
its current working directory to its data directory when it begins
executing.)
If you specify
innodb_data_home_dir
as an empty
string, you can specify absolute paths for the data files listed
in the innodb_data_file_path
value. The following example is equivalent to the preceding one:
[mysqld] innodb_data_home_dir = innodb_data_file_path=/ibdata/ibdata1:50M;/ibdata/ibdata2:50M:autoextend
Specifying InnoDB Configuration Options
Sample my.cnf
file for
small systems. Suppose that you have a computer with
512MB RAM and one hard disk. The following example shows possible
configuration parameters in my.cnf
or
my.ini
for InnoDB
,
including the autoextend
attribute. The example
suits most users, both on Unix and Windows, who do not want to
distribute InnoDB
data files and log files onto
several disks. It creates an auto-extending data file
ibdata1
and two InnoDB
log
files ib_logfile0
and
ib_logfile1
in the MySQL data directory.
[mysqld] # You can write your other MySQL server options here # ... # Data files must be able to hold your data and indexes. # Make sure that you have enough free disk space. innodb_data_file_path = ibdata1:10M:autoextend # # Set buffer pool size to 50-80% of your computer's memory innodb_buffer_pool_size=256M innodb_additional_mem_pool_size=20M # # Set the log file size to about 25% of the buffer pool size innodb_log_file_size=64M innodb_log_buffer_size=8M # innodb_flush_log_at_trx_commit=1
Note that data files must be less than 2GB in some file systems. The combined size of the log files must be less than 4GB. The combined size of data files must be at least 10MB.
Setting Up the InnoDB System Tablespace
When you create an InnoDB
system tablespace for
the first time, it is best that you start the MySQL server from
the command prompt. InnoDB
then prints the
information about the database creation to the screen, so you can
see what is happening. For example, on Windows, if
mysqld is located in C:\Program
Files\MySQL\MySQL Server 5.5\bin
, you can
start it like this:
C:\> "C:\Program Files\MySQL\MySQL Server 5.5\bin\mysqld" --console
If you do not send server output to the screen, check the server's
error log to see what InnoDB
prints during the
startup process.
For an example of what the information displayed by
InnoDB
should look like, see
Section 13.3.3.2, “Creating the InnoDB
Tablespace”.
Editing the MySQL Configuration File
You can place InnoDB
options in the
[mysqld]
group of any option file that your
server reads when it starts. The locations for option files are
described in Section 4.2.3.3, “Using Option Files”.
If you installed MySQL on Windows using the installation and
configuration wizards, the option file will be the
my.ini
file located in your MySQL
installation directory. See
Section 2.3.5.1, “Starting the MySQL Server Instance Configuration Wizard”.
If your PC uses a boot loader where the C:
drive is not the boot drive, your only option is to use the
my.ini
file in your Windows directory
(typically C:\WINDOWS
). You can use the
SET
command at the command prompt in a console
window to print the value of WINDIR
:
C:\> SET WINDIR
windir=C:\WINDOWS
To make sure that mysqld reads options only
from a specific file, use the
--defaults-file
option as the
first option on the command line when starting the server:
mysqld --defaults-file=your_path_to_my_cnf
Sample my.cnf
file for
large systems. Suppose that you have a Linux computer
with 2GB RAM and three 60GB hard disks at directory paths
/
, /dr2
and
/dr3
. The following example shows possible
configuration parameters in my.cnf
for
InnoDB
.
[mysqld] # You can write your other MySQL server options here # ... innodb_data_home_dir = # # Data files must be able to hold your data and indexes innodb_data_file_path = /db/ibdata1:2000M;/dr2/db/ibdata2:2000M:autoextend # # Set buffer pool size to 50-80% of your computer's memory, # but make sure on Linux x86 total memory usage is < 2GB innodb_buffer_pool_size=1G innodb_additional_mem_pool_size=20M innodb_log_group_home_dir = /dr3/iblogs # # Set the log file size to about 25% of the buffer pool size innodb_log_file_size=250M innodb_log_buffer_size=8M # innodb_flush_log_at_trx_commit=1 innodb_lock_wait_timeout=50 # # Uncomment the next line if you want to use it #innodb_thread_concurrency=5
Determining the Maximum Memory Allocation for InnoDB
On 32-bit GNU/Linux x86, be careful not to set memory usage too
high. glibc
may permit the process heap to
grow over thread stacks, which crashes your server. It is a risk
if the value of the following expression is close to or exceeds
2GB:
innodb_buffer_pool_size + key_buffer_size + max_connections*(sort_buffer_size+read_buffer_size+binlog_cache_size) + max_connections*2MB
Each thread uses a stack (often 2MB, but only 256KB in MySQL
binaries provided by Oracle Corporation.) and in the worst case
also uses sort_buffer_size + read_buffer_size
additional memory.
Tuning other mysqld server parameters. The following values are typical and suit most users:
[mysqld]
skip-external-locking
max_connections=200
read_buffer_size=1M
sort_buffer_size=1M
#
# Set key_buffer to 5 - 50% of your RAM depending on how much
# you use MyISAM tables, but keep key_buffer_size + InnoDB
# buffer pool size < 80% of your RAM
key_buffer_size=value
On Linux, if the kernel is enabled for large page support,
InnoDB
can use large pages to allocate memory
for its buffer pool and additional memory pool. See
Section 7.11.4.2, “Enabling Large Page Support”.
Turning Off InnoDB
Oracle recommends InnoDB as the preferred storage engine for typical database applications, from single-user wikis and blogs running on a local system, to high-end applications pushing the limits of performance. In MySQL 5.5, InnoDB is is the default storage engine for new tables.
If you do not want to use InnoDB
tables, start
the server with the
--innodb=OFF
or
--skip-innodb
option to disable the InnoDB
storage engine. In
this case, because the default storage engine is
InnoDB
, the server will not start
unless you also use
--default-storage-engine
to set the
default to some other engine.
By default, all InnoDB tables and indexes are stored in the
system tablespace.
As an alternative, you can store each InnoDB
table and its indexes in its own file. This feature is called
“multiple tablespaces” because each table that is
created when this setting is in effect has its own tablespace.
Using multiple tablespaces is useful in a number of situations:
You can back up or restore a single table quickly without interrupting the use of other
InnoDB
tables, using the MySQL Enterprise Backup product. See Restoring a Single.ibd
File for the procedure and restrictions.Storing specific tables on separate physical disks, for I/O optimization or backup purposes.
Restoring backups of single tables quickly without interrupting the use of other
InnoDB
tables.Using compressed row format to compress table data.
Reclaiming disk space when truncating a table.
Enabling and Disabling Multiple Tablespaces
To enable multiple tablespaces, start the server with the
--innodb_file_per_table
option. For
example, add a line to the [mysqld]
section of
my.cnf
:
[mysqld] innodb_file_per_table
With multiple tablespaces enabled, InnoDB
stores each newly created table in its own
file
in the appropriate database directory. Unlike the
tbl_name
.ibdMyISAM
storage engine, with its separate
and
tbl_name
.MYD
files
for indexes and data, tbl_name
.MYIInnoDB
stores the data
and the indexes together in a single .ibd
file. The
file
is still created as usual.
tbl_name
.frm
If you remove the
innodb_file_per_table
line from
my.cnf
and restart the server,
InnoDB
creates any new tables inside the shared
tablespace files.
You can always access both tables in the system tablespace and
tables in their own tablespaces, regardless of the file-per-table
setting. To move a table from the system tablespace to its own
tablespace, or vice versa, you can change the
innodb_file_per_table
setting and
issue the command:
-- Move table from system tablespace to its own tablespace. SET GLOBAL innodb_file_per_table=1; ALTER TABLEtable_name
ENGINE=InnoDB; -- Move table from its own tablespace to system tablespace. SET GLOBAL innodb_file_per_table=0; ALTER TABLEtable_name
ENGINE=InnoDB;
InnoDB
always needs the shared tablespace
because it puts its internal data dictionary and undo logs
there. The .ibd
files are not sufficient
for InnoDB
to operate.
When a table is moved out of the system tablespace into its own
.ibd
file, the data files that make up the
system tablespace remain the same size. The space formerly
occupied by the table can be reused for new
InnoDB
data, but is not reclaimed for use by
the operating system. When moving large
InnoDB
tables out of the system tablespace,
where disk space is limited, you might prefer to turn on
innodb_file_per_table
and then recreate the
entire instance using the mysqldump command.
Portability Considerations for .ibd Files
You cannot freely move .ibd
files between
database directories as you can with MyISAM
table files. The table definition stored in the
InnoDB
shared tablespace includes the database
name. The transaction IDs and log sequence numbers stored in the
tablespace files also differ between databases.
To move an .ibd
file and the associated table
from one database to another, use a RENAME
TABLE
statement:
RENAME TABLEdb1.tbl_name
TOdb2.tbl_name
;
If you have a “clean” backup of an
.ibd
file, you can restore it to the MySQL
installation from which it originated as follows:
The table must not have been dropped or truncated since you copied the
.ibd
file, because doing so changes the table ID stored inside the tablespace.Issue this
ALTER TABLE
statement to delete the current.ibd
file:ALTER TABLE
tbl_name
DISCARD TABLESPACE;Copy the backup
.ibd
file to the proper database directory.Issue this
ALTER TABLE
statement to tellInnoDB
to use the new.ibd
file for the table:ALTER TABLE
tbl_name
IMPORT TABLESPACE;
In this context, a “clean” .ibd
file backup is one for which the following requirements are
satisfied:
There are no uncommitted modifications by transactions in the
.ibd
file.There are no unmerged insert buffer entries in the
.ibd
file.Purge has removed all delete-marked index records from the
.ibd
file.mysqld has flushed all modified pages of the
.ibd
file from the buffer pool to the file.
You can make a clean backup .ibd
file using
the following method:
Stop all activity from the mysqld server and commit all transactions.
Wait until
SHOW ENGINE INNODB STATUS
shows that there are no active transactions in the database, and the main thread status ofInnoDB
isWaiting for server activity
. Then you can make a copy of the.ibd
file.
Another method for making a clean copy of an
.ibd
file is to use the MySQL Enterprise
Backup product:
Use MySQL Enterprise Backup to back up the
InnoDB
installation.Start a second mysqld server on the backup and let it clean up the
.ibd
files in the backup.
You can use raw disk partitions as data files in the system tablespace. Using a raw disk, you can perform nonbuffered I/O on Windows and on some Unix systems without file system overhead. Perform tests with and without raw partitions to verify whether this change actually improves performance on your system.
When you create a new data file, put the keyword
newraw
immediately after the data file size
in innodb_data_file_path
. The
partition must be at least as large as the size that you
specify. Note that 1MB in InnoDB
is 1024
× 1024 bytes, whereas 1MB in disk specifications usually
means 1,000,000 bytes.
[mysqld] innodb_data_home_dir= innodb_data_file_path=/dev/hdd1:3Gnewraw;/dev/hdd2:2Gnewraw
The next time you start the server, InnoDB
notices the newraw
keyword and initializes
the new partition. However, do not create or change any
InnoDB
tables yet. Otherwise, when you next
restart the server, InnoDB
reinitializes the
partition and your changes are lost. (As a safety measure
InnoDB
prevents users from modifying data
when any partition with newraw
is specified.)
After InnoDB
has initialized the new
partition, stop the server, change newraw
in
the data file specification to raw
:
[mysqld] innodb_data_home_dir= innodb_data_file_path=/dev/hdd1:3Graw;/dev/hdd2:2Graw
Then restart the server and InnoDB
permits
changes to be made.
On Windows, you can allocate a disk partition as a data file like this:
[mysqld] innodb_data_home_dir= innodb_data_file_path=//./D::10Gnewraw
The //./
corresponds to the Windows syntax
of \\.\
for accessing physical drives.
When you use a raw disk partition, be sure that it has
permissions that enable read and write access by the account
used for running the MySQL server. For example, if you run the
server as the mysql
user, the partition must
permit read and write access to mysql
. If you
run the server with the --memlock
option, the server must be run as root
, so
the partition must permit access to root
.
Suppose that you have installed MySQL and have edited your
option file so that it contains the necessary
InnoDB
configuration parameters. Before
starting MySQL, verify that the directories you have specified
for InnoDB
data files and log files exist and
that the MySQL server has access rights to those directories.
InnoDB
does not create directories, only
files. Check also that you have enough disk space for the data
and log files.
It is best to run the MySQL server mysqld
from the command prompt when you first start the server with
InnoDB
enabled, not from
mysqld_safe or as a Windows service. When you
run from a command prompt you see what mysqld
prints and what is happening. On Unix, just invoke
mysqld. On Windows, start
mysqld with the
--console
option to direct the
output to the console window.
When you start the MySQL server after initially configuring
InnoDB
in your option file,
InnoDB
creates your data files and log files,
and prints something like this:
InnoDB: The first specified datafile /home/heikki/data/ibdata1 did not exist: InnoDB: a new database to be created! InnoDB: Setting file /home/heikki/data/ibdata1 size to 134217728 InnoDB: Database physically writes the file full: wait... InnoDB: datafile /home/heikki/data/ibdata2 did not exist: new to be created InnoDB: Setting file /home/heikki/data/ibdata2 size to 262144000 InnoDB: Database physically writes the file full: wait... InnoDB: Log file /home/heikki/data/logs/ib_logfile0 did not exist: new to be created InnoDB: Setting log file /home/heikki/data/logs/ib_logfile0 size to 5242880 InnoDB: Log file /home/heikki/data/logs/ib_logfile1 did not exist: new to be created InnoDB: Setting log file /home/heikki/data/logs/ib_logfile1 size to 5242880 InnoDB: Doublewrite buffer not found: creating new InnoDB: Doublewrite buffer created InnoDB: Creating foreign key constraint system tables InnoDB: Foreign key constraint system tables created InnoDB: Started mysqld: ready for connections
At this point InnoDB
has initialized its
tablespace and log files. You can connect to the MySQL server
with the usual MySQL client programs like
mysql. When you shut down the MySQL server
with mysqladmin shutdown, the output is like
this:
010321 18:33:34 mysqld: Normal shutdown 010321 18:33:34 mysqld: Shutdown Complete InnoDB: Starting shutdown... InnoDB: Shutdown completed
You can look at the data file and log directories and you see the files created there. When MySQL is started again, the data files and log files have been created already, so the output is much briefer:
InnoDB: Started mysqld: ready for connections
If you add the
innodb_file_per_table
option to
my.cnf
, InnoDB
stores
each table in its own .ibd
file in the same
MySQL database directory where the .frm
file is created. See
Section 13.3.3, “Using Per-Table Tablespaces”.
The troubleshooting steps for InnoDB
I/O
problems depend on when the problem occurs: during startup of
the MySQL server, or during normal operations when a DML or DDL
statement fails due to problems at the file system level.
Initialization Problems
If something goes wrong when InnoDB
attempts
to initialize its tablespace or its log files, delete all files
created by InnoDB
: all
ibdata
files and all
ib_logfile
files. If you already created
some InnoDB
tables, also delete the
corresponding .frm
files for these tables,
and any .ibd
files if you are using
multiple tablespaces, from the MySQL database directories. Then
try the InnoDB
database creation again. For
easiest troubleshooting, start the MySQL server from a command
prompt so that you see what is happening.
Runtime Problems
If InnoDB
prints an operating system error
during a file operation, usually the problem has one of the
following solutions:
Make sure the
InnoDB
data file directory and theInnoDB
log directory exist.Make sure mysqld has access rights to create files in those directories.
Make sure mysqld can read the proper
my.cnf
ormy.ini
option file, so that it starts with the options that you specified.Make sure the disk is not full and you are not exceeding any disk quota.
Make sure that the names you specify for subdirectories and data files do not clash.
Doublecheck the syntax of the
innodb_data_home_dir
andinnodb_data_file_path
values. In particular, anyMAX
value in theinnodb_data_file_path
option is a hard limit, and exceeding that limit causes a fatal error.
This section describes the InnoDB
-related
command options and system variables. System variables that are
true or false can be enabled at server startup by naming them, or
disabled by using a --skip-
prefix. For
example, to enable or disable InnoDB
checksums,
you can use --innodb_checksums
or
--skip-innodb_checksums
on the command line, or
innodb_checksums
or
skip-innodb_checksums
in an option file. System
variables that take a numeric value can be specified as
--
on the command line or as
var_name
=value
in option files. For more information on specifying options and
system variables, see Section 4.2.3, “Specifying Program Options”. Many of
the system variables can be changed at runtime (see
Section 5.1.4.2, “Dynamic System Variables”).
var_name
=value
Certain options control the locations and layout of the
InnoDB
data files.
Section 13.3.2, “Configuring InnoDB
” explains how to use these
options. Many other options, that you might not use initially,
help to tune InnoDB
performance characteristics
based on machine capacity and your database
workload. The
performance-related options are explained in
Section 13.3.14, “InnoDB
Performance Tuning and Troubleshooting” and
Section 13.4.7, “InnoDB Performance and Scalability Enhancements”.
Table 13.3. InnoDB
Option/Variable
Reference
InnoDB
Command Options
Command-Line Format --ignore-builtin-innodb
Option-File Format ignore-builtin-innodb
Option Sets Variable Yes, ignore_builtin_innodb
Variable Name ignore-builtin-innodb
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
This option causes the server to behave as if the built-in
InnoDB
is not present. It has these effects:Other
InnoDB
options (including--innodb
and--skip-innodb
) will not be recognized and should not be used.The server will not start if the default storage engine is set to
InnoDB
. Use--default-storage-engine
to set the default to some other engine if necessary.InnoDB
will not appear in the output ofSHOW ENGINES
.
Controls loading of the
InnoDB
storage engine, if the server was compiled withInnoDB
support. This option has a tristate format, with possible values ofOFF
,ON
, orFORCE
. See Section 5.1.7.1, “Installing and Uninstalling Plugins”.To disable
InnoDB
, use--innodb=OFF
or--skip-innodb
. In this case, because the default storage engine isInnoDB
, the server will not start unless you also use--default-storage-engine
to set the default to some other engine.Command-Line Format --innodb-status-file
Option-File Format innodb-status-file
Variable Name innodb-status-file
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
Controls whether
InnoDB
creates a file namedinnodb_status.
in the MySQL data directory. If enabled,<pid>
InnoDB
periodically writes the output ofSHOW ENGINE INNODB STATUS
to this file.By default, the file is not created. To create it, start mysqld with the
--innodb-status-file=1
option. The file is deleted during normal shutdown.Disable the
InnoDB
storage engine. See the description of--innodb
.
InnoDB
System Variables
Whether the server was started with the
--ignore-builtin-innodb
option, which causes the server to behave as if the built-inInnoDB
is not present. For more information, see the description of--ignore-builtin-innodb
under “InnoDB
Command Options” earlier in this section.Command-Line Format --innodb_adaptive_flushing=#
Option-File Format innodb_adaptive_flushing
Option Sets Variable Yes, innodb_adaptive_flushing
Variable Name innodb_adaptive_flushing
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default ON
Specifies whether to dynamically adjust the rate of flushing dirty pages in the
InnoDB
buffer pool based on the workload. Adjusting the flush rate dynamically is intended to avoid bursts of I/O activity. This setting is enabled by default.Command-Line Format --innodb_adaptive_hash_index=#
Option-File Format innodb_adaptive_hash_index
Option Sets Variable Yes, innodb_adaptive_hash_index
Variable Name innodb_adaptive_hash_index
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default ON
Whether InnoDB adaptive hash indexes are enabled or disabled (see Section 13.3.11.4, “Adaptive Hash Indexes”). This variable is enabled by default. Use
--skip-innodb_adaptive_hash_index
at server startup to disable it.innodb_additional_mem_pool_size
Command-Line Format --innodb_additional_mem_pool_size=#
Option-File Format innodb_additional_mem_pool_size
Option Sets Variable Yes, innodb_additional_mem_pool_size
Variable Name innodb_additional_mem_pool_size
Variable Scope Global Dynamic Variable No Deprecated 5.6.3 Permitted Values Type numeric
Default 8388608
Range 2097152 .. 4294967295
The size in bytes of a memory pool
InnoDB
uses to store data dictionary information and other internal data structures. The more tables you have in your application, the more memory you need to allocate here. IfInnoDB
runs out of memory in this pool, it starts to allocate memory from the operating system and writes warning messages to the MySQL error log. The default value is 8MB.Command-Line Format --innodb_autoextend_increment=#
Option-File Format innodb_autoextend_increment
Option Sets Variable Yes, innodb_autoextend_increment
Variable Name innodb_autoextend_increment
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 8
Range 1 .. 1000
The increment size (in MB) for extending the size of an auto-extending shared tablespace file when it becomes full. The default value is 8. This variable does not affect the per-table tablespace files that are created if you use
innodb_file_per_table=1
. Those files are auto-extending regardless of the value ofinnodb_autoextend_increment
. The initial extensions are by small amounts, after which extensions occur in increments of 4MB.Command-Line Format --innodb_autoinc_lock_mode=#
Option-File Format innodb_autoinc_lock_mode
Option Sets Variable Yes, innodb_autoinc_lock_mode
Variable Name innodb_autoinc_lock_mode
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 1
Valid Values 0
1
2
The locking mode to use for generating auto-increment values. The permissible values are 0, 1, or 2, for “traditional”, “consecutive”, or “interleaved” lock mode, respectively. Section 13.3.5.3, “
AUTO_INCREMENT
Handling inInnoDB
”, describes the characteristics of these modes.This variable has a default of 1 (“consecutive” lock mode).
Version Introduced 5.5.4 Command-Line Format --innodb_buffer_pool_instances=#
Option-File Format innodb_buffer_pool_instances
Option Sets Variable Yes, innodb_buffer_pool_instances
Variable Name innodb_buffer_pool_instances
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 1
Range 1 .. 64
The number of regions that the
InnoDB
buffer pool is divided into. For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, by reducing contention as different threads read and write to cached pages. Each page that is stored in or read from the buffer pool is assigned to one of the buffer pool instances randomly, using a hashing function. Each buffer pool manages its own free lists, flush lists, LRUs, and all other data structures connected to a buffer pool, and is protected by its own buffer pool mutex.This option only takes effect when you set the
innodb_buffer_pool_size
to a size of 1 gigabyte or more. The total size you specify is divided up among all the buffer pools. We recommend specifying a combination ofinnodb_buffer_pool_instances
andinnodb_buffer_pool_size
so that each buffer pool instance is at least 1 gigabyte.Command-Line Format --innodb_buffer_pool_size=#
Option-File Format innodb_buffer_pool_size
Option Sets Variable Yes, innodb_buffer_pool_size
Variable Name innodb_buffer_pool_size
Variable Scope Global Dynamic Variable No Permitted Values Platform Bit Size 32
Type numeric
Default 134217728
Range 1048576 .. 2**32-1
The size in bytes of the memory buffer
InnoDB
uses to cache data and indexes of its tables. The default value is 128MB, increased from a historical default of 8MB. The maximum value depends on the CPU architecture, 32-bit or 64-bit. For 32-bit systems, the CPU architecture and operating system sometimes impose a lower practical maximum size.The larger you set this value, the less disk I/O is needed to access data in tables. On a dedicated database server, you may set this to up to 80% of the machine physical memory size. Be prepared to scale back this value if these other issues occur:
Competition for physical memory might cause paging in the operating system.
InnoDB reserves additional memory for buffers and control structures, so that the total allocated space is approximately 10% greater than the specified size.
The address space must be contiguous, which can be an issue on Windows systems with DLLs that load at specific addresses.
The time to initialize the buffer pool is roughly proportional to its size. On large installations, this initialization time may be significant. For example, on a modern Linux x86_64 server, initialization of a 10GB buffer pool takes approximately 6 seconds. See Section 7.9.1, “The
InnoDB
Buffer Pool”.
Command-Line Format --innodb_change_buffering=#
Option-File Format innodb_change_buffering
Option Sets Variable Yes, innodb_change_buffering
Variable Name innodb_change_buffering
Variable Scope Global Dynamic Variable Yes Permitted Values (<= 5.5.3) Type enumeration
Default inserts
Valid Values inserts
none
Permitted Values (>= 5.5.4) Type enumeration
Default all
Valid Values inserts
deletes
purges
changes
all
none
Whether
InnoDB
performs change buffering, an optimization that delays write operations to secondary indexes so that the I/O operations can be performed sequentially. The permitted values areinserts
(buffer insert operations),deletes
(buffer delete operations; strictly speaking, the writes that mark index records for later deletion during a purge operation),changes
(buffer insert and delete-marking operations),purges
(buffer purge operations, the writes when deleted index entries are finally garbage-collected),all
(buffer insert, delete-marking, and purge operations) andnone
(do not buffer any operations). The default isall
. For details, see Section 13.4.7.4, “Controlling InnoDB Change Buffering”.Command-Line Format --innodb_checksums
Option-File Format innodb_checksums
Option Sets Variable Yes, innodb_checksums
Variable Name innodb_checksums
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default ON
InnoDB
can use checksum validation on all pages read from the disk to ensure extra fault tolerance against broken hardware or data files. This validation is enabled by default. However, under some rare circumstances (such as when running benchmarks) this extra safety feature is unneeded and can be disabled with--skip-innodb-checksums
.Command-Line Format --innodb_commit_concurrency=#
Option-File Format innodb_commit_concurrency
Option Sets Variable Yes, innodb_commit_concurrency
Variable Name innodb_commit_concurrency
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
Range 0 .. 1000
The number of threads that can commit at the same time. A value of 0 (the default) permits any number of transactions to commit simultaneously.
The value of
innodb_commit_concurrency
cannot be changed at runtime from zero to nonzero or vice versa. The value can be changed from one nonzero value to another.Command-Line Format --innodb_concurrency_tickets=#
Option-File Format innodb_concurrency_tickets
Option Sets Variable Yes, innodb_concurrency_tickets
Variable Name innodb_concurrency_tickets
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 500
Range 1 .. 4294967295
The number of threads that can enter
InnoDB
concurrently is determined by theinnodb_thread_concurrency
variable. A thread is placed in a queue when it tries to enterInnoDB
if the number of threads has already reached the concurrency limit. When a thread is permitted to enterInnoDB
, it is given a number of “free tickets” equal to the value ofinnodb_concurrency_tickets
, and the thread can enter and leaveInnoDB
freely until it has used up its tickets. After that point, the thread again becomes subject to the concurrency check (and possible queuing) the next time it tries to enterInnoDB
. The default value is 500.Command-Line Format --innodb_data_file_path=name
Option-File Format innodb_data_file_path
Variable Name innodb_data_file_path
Variable Scope Global Dynamic Variable No Permitted Values Type file name
The paths to individual data files and their sizes. The full directory path to each data file is formed by concatenating
innodb_data_home_dir
to each path specified here. The file sizes are specified in KB, MB, or GB (1024MB) by appendingK
,M
, orG
to the size value. The sum of the sizes of the files must be at least 10MB. If you do not specifyinnodb_data_file_path
, the default behavior is to create a single 10MB auto-extending data file namedibdata1
. The size limit of individual files is determined by your operating system. You can set the file size to more than 4GB on those operating systems that support big files. You can also use raw disk partitions as data files. For detailed information on configuringInnoDB
tablespace files, see Section 13.3.2, “ConfiguringInnoDB
”.Command-Line Format --innodb_data_home_dir=path
Option-File Format innodb_data_home_dir
Option Sets Variable Yes, innodb_data_home_dir
Variable Name innodb_data_home_dir
Variable Scope Global Dynamic Variable No Permitted Values Type file name
The common part of the directory path for all
InnoDB
data files in the shared tablespace. This setting does not affect the location of per-file tablespaces wheninnodb_file_per_table
is enabled. The default value is the MySQL data directory. If you specify the value as an empty string, you can use absolute file paths ininnodb_data_file_path
.Command-Line Format --innodb-doublewrite
Option-File Format innodb_doublewrite
Option Sets Variable Yes, innodb_doublewrite
Variable Name innodb_doublewrite
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
If this variable is enabled (the default),
InnoDB
stores all data twice, first to the doublewrite buffer, and then to the actual data files. This variable can be turned off with--skip-innodb_doublewrite
for benchmarks or cases when top performance is needed rather than concern for data integrity or possible failures.Command-Line Format --innodb_fast_shutdown[=#]
Option-File Format innodb_fast_shutdown
Option Sets Variable Yes, innodb_fast_shutdown
Variable Name innodb_fast_shutdown
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 1
Valid Values 0
1
2
The
InnoDB
shutdown mode. If the value is 0,InnoDB
does a slow shutdown, a full purge and an insert buffer merge before shutting down. If the value is 1 (the default),InnoDB
skips these operations at shutdown, a process known as a fast shutdown. If the value is 2,InnoDB
flushes its logs and shuts down cold, as if MySQL had crashed; no committed transactions are lost, but the crash recovery operation makes the next startup take longer.The slow shutdown can take minutes, or even hours in extreme cases where substantial amounts of data are still buffered. Use the slow shutdown technique before upgrading or downgrading between MySQL major releases, so that all data files are fully prepared in case the upgrade process updates the file format.
Use
innodb_fast_shutdown=2
in emergency or troubleshooting situations, to get the absolute fastest shutdown if data is at risk of corruption.Command-Line Format --innodb_file_format=#
Option-File Format innodb_file_format
Option Sets Variable Yes, innodb_file_format
Variable Name innodb_file_format
Variable Scope Global Dynamic Variable Yes Permitted Values (>= 5.5.0, <= 5.5.6) Type string
Default Barracuda
Valid Values Antelope
Barracuda
Permitted Values (>= 5.5.7) Type string
Default Antelope
Valid Values Antelope
Barracuda
The file format to use for new
InnoDB
tables. Currently,Antelope
andBarracuda
are supported. This applies only for tables that have their own tablespace, so for it to have an effect,innodb_file_per_table
must be enabled. The Barracuda file format is required for certain InnoDB features such as table compression.Command-Line Format --innodb_file_format_check=#
Option-File Format innodb_file_format_check
Option Sets Variable Yes, innodb_file_format_check
Variable Name innodb_file_format_check
Variable Scope Global Dynamic Variable No Permitted Values (<= 5.5.0) Type string
Default Antelope
Permitted Values (>= 5.5.4) Type string
Default Barracuda
Permitted Values (>= 5.5.5) Type boolean
Default ON
As of MySQL 5.5.5, this variable can be set to 1 or 0 at server startup to enable or disable whether
InnoDB
checks the file format tag in the shared tablespace (for example,Antelope
orBarracuda
). If the tag is checked and is higher than that supported by the current version ofInnoDB
, an error occurs andInnoDB
does not start. If the tag is not higher,InnoDB
sets the value ofinnodb_file_format_max
to the file format tag.Before MySQL 5.5.5, this variable can be set to 1 or 0 at server startup to enable or disable whether
InnoDB
checks the file format tag in the shared tablespace. If the tag is checked and is higher than that supported by the current version ofInnoDB
, an error occurs andInnoDB
does not start. If the tag is not higher,InnoDB
sets the value ofinnodb_file_format_check
to the file format tag, which is the value seen at runtime.Version Introduced 5.5.5 Command-Line Format --innodb_file_format_max=#
Option-File Format innodb_file_format_max
Option Sets Variable Yes, innodb_file_format_max
Variable Name innodb_file_format_max
Variable Scope Global Dynamic Variable Yes Permitted Values Type string
Default Antelope
Valid Values Antelope
Barracuda
At server startup,
InnoDB
sets the value ofinnodb_file_format_max
to the file format tag in the shared tablespace (for example,Antelope
orBarracuda
). If the server creates or opens a table with a “higher” file format, it sets the value ofinnodb_file_format_max
to that format.This variable was added in MySQL 5.5.5.
Command-Line Format --innodb_file_per_table
Option-File Format innodb_file_per_table
Variable Name innodb_file_per_table
Variable Scope Global Dynamic Variable Yes Permitted Values (>= 5.5.0, <= 5.5.6) Type boolean
Default ON
Permitted Values (>= 5.5.7) Type boolean
Default OFF
If
innodb_file_per_table
is disabled (the default),InnoDB
creates tables in the system tablespace. Ifinnodb_file_per_table
is enabled,InnoDB
creates each new table using its own.ibd
file for storing data and indexes, rather than in the system tablespace. See Section 13.3.3, “Using Per-Table Tablespaces” for information about the features, such asInnoDB
table compression, that only work for tables stored in separate tablespaces.innodb_flush_log_at_trx_commit
Command-Line Format --innodb_flush_log_at_trx_commit[=#]
Option-File Format innodb_flush_log_at_trx_commit
Option Sets Variable Yes, innodb_flush_log_at_trx_commit
Variable Name innodb_flush_log_at_trx_commit
Variable Scope Global Dynamic Variable Yes Permitted Values Type enumeration
Default 1
Valid Values 0
1
2
If the value of
innodb_flush_log_at_trx_commit
is 0, the log buffer is written out to the log file once per second and the flush to disk operation is performed on the log file, but nothing is done at a transaction commit. When the value is 1 (the default), the log buffer is written out to the log file at each transaction commit and the flush to disk operation is performed on the log file. When the value is 2, the log buffer is written out to the file at each commit, but the flush to disk operation is not performed on it. However, the flushing on the log file takes place once per second also when the value is 2. Note that the once-per-second flushing is not 100% guaranteed to happen every second, due to process scheduling issues.The default value of 1 is required for full ACID compliance. You can achieve better performance by setting the value different from 1, but then you can lose up to one second worth of transactions in a crash. With a value of 0, any mysqld process crash can erase the last second of transactions. With a value of 2, only an operating system crash or a power outage can erase the last second of transactions.
InnoDB
's crash recovery works regardless of the value.For the greatest possible durability and consistency in a replication setup using
InnoDB
with transactions, useinnodb_flush_log_at_trx_commit=1
andsync_binlog=1
in your master servermy.cnf
file.CautionMany operating systems and some disk hardware fool the flush-to-disk operation. They may tell mysqld that the flush has taken place, even though it has not. Then the durability of transactions is not guaranteed even with the setting 1, and in the worst case a power outage can even corrupt the
InnoDB
database. Using a battery-backed disk cache in the SCSI disk controller or in the disk itself speeds up file flushes, and makes the operation safer. You can also try using the Unix command hdparm to disable the caching of disk writes in hardware caches, or use some other command specific to the hardware vendor.Command-Line Format --innodb_flush_method=name
Option-File Format innodb_flush_method
Option Sets Variable Yes, innodb_flush_method
Variable Name innodb_flush_method
Variable Scope Global Dynamic Variable No Permitted Values Type (solaris) enumeration
Default fdatasync
Valid Values O_DSYNC
O_DIRECT
By default,
InnoDB
uses thefsync()
system call to flush both the data and log files. Ifinnodb_flush_method
option is set toO_DSYNC
,InnoDB
usesO_SYNC
to open and flush the log files, andfsync()
to flush the data files. IfO_DIRECT
is specified (available on some GNU/Linux versions, FreeBSD, and Solaris),InnoDB
usesO_DIRECT
(ordirectio()
on Solaris) to open the data files, and usesfsync()
to flush both the data and log files. Note thatInnoDB
usesfsync()
instead offdatasync()
, and it does not useO_DSYNC
by default because there have been problems with it on many varieties of Unix. This variable is relevant only for Unix. On Windows, the flush method is alwaysasync_unbuffered
and cannot be changed.Depending on hardware configuration, setting
innodb_flush_method
toO_DIRECT
can either have either a positive or negative effect on performance. Benchmark your particular configuration to decide which setting to use. The mix of read and write operations in your workload can also affect which setting performs better for you. For example, on a system with a hardware RAID controller and battery-backed write cache,O_DIRECT
can help to avoid double buffering between theInnoDB
buffer pool and the operating system's filesystem cache. On some systems whereInnoDB
data and log files are located on a SAN, the default value orO_DSYNC
might be faster for a read-heavy workload with mostlySELECT
statements. Always test this parameter with the same type of hardware and workload that reflects your production environment.Formerly, a value of
fdatasync
also specified the default behavior. This value was removed, due to confusion that a value offdatasync
causedfsync()
system calls rather thanfdatasync()
for flushing. To obtain the default value now, do not set any value forinnodb_flush_method
at startup.Command-Line Format --innodb_force_recovery=#
Option-File Format innodb_force_recovery
Option Sets Variable Yes, innodb_force_recovery
Variable Name innodb_force_recovery
Variable Scope Global Dynamic Variable No Permitted Values Type enumeration
Default 0
Valid Values 0
1
2
3
4
5
6
The crash recovery mode. Possible values are from 0 to 6. The meanings of these values are described in Section 13.3.7.2, “Forcing
InnoDB
Recovery”.WarningOnly set this variable greater than 0 in an emergency situation, to dump your tables from a corrupt database. As a safety measure,
InnoDB
prevents any changes to its data when this variable is greater than 0. This restriction also prohibits some queries that useWHERE
orORDER BY
clauses, because high values can prevent queries from using indexes.Command-Line Format --innodb_io_capacity=#
Option-File Format innodb_io_capacity
Option Sets Variable Yes, innodb_io_capacity
Variable Name innodb_io_capacity
Variable Scope Global Dynamic Variable Yes Permitted Values Platform Bit Size 32
Type numeric
Default 200
Range 100 .. 2**32-1
Permitted Values Platform Bit Size 64
Type numeric
Default 200
Range 100 .. 2**64-1
An upper limit on the I/O activity performed by the
InnoDB
background tasks, such as flushing pages from the buffer pool and merging data from the insert buffer. The default value is 200. For busy systems capable of higher I/O rates, you can set a higher value at server startup, to help the server handle the background maintenance work associated with a high rate of row changes. For systems with individual 5400 RPM or 7200 RPM drives, you might lower the value to the former default of100
.This parameter should be set to approximately the number of I/O operations that the system can perform per second. Ideally, keep this setting as low as practical, but not so low that these background activities fall behind. If the value is too high, data is removed from the buffer pool and insert buffer too quickly to provide significant benefit from the caching.
The value represents an estimated proportion of the I/O operations per second (IOPS) available to older-generation disk drives that could perform about 100 IOPS. The current default of 200 reflects that modern storage devices are capable of much higher I/O rates.
In general, you can increase the value as a function of the number of drives used for
InnoDB
I/O, particularly fast drives capable of high numbers of IOPS. For example, systems that use multiple disks or solid-state disks forInnoDB
are likely to benefit from the ability to control this parameter.Although you can specify a very high number, in practice such large values cease to have any benefit; for example, a value of one million would be considered very high.
You can set the
innodb_io_capacity
value to any number 100 or greater, and the default value is200
. You can set the value of this parameter in the MySQL option file (my.cnf
ormy.ini
) or change it dynamically with theSET GLOBAL
command, which requires theSUPER
privilege.See Section 13.4.7.11, “Controlling the Master Thread I/O Rate” for more guidelines about this option. For general information about InnoDB I/O performance, see Section 7.5.7, “Optimizing
InnoDB
Disk I/O”.Version Introduced 5.5.14 Command-Line Format --innodb_large_prefix
Option-File Format innodb_large_prefix
Option Sets Variable Yes, innodb_large_prefix
Variable Name innodb_large_prefix
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default OFF
Enable this option to allow index key prefixes longer than 767 bytes (up to 3072 bytes), for
InnoDB
tables that use theDYNAMIC
andCOMPRESSED
row formats. (Creating such tables also requires the option valuesinnodb_file_format=barracuda
andinnodb_file_per_table=true
.) See Section 13.3.15, “Limits onInnoDB
Tables” for the relevant maximums associated with index key prefixes under various settings.For tables using the
REDUNDANT
andCOMPACT
row formats, this option does not affect the allowed key prefix length. It does introduce a new error possibility. When this setting is enabled, attempting to create an index prefix with a key length greater than 3072 for aREDUNDANT
orCOMPACT
table causes an errorER_INDEX_COLUMN_TOO_LONG
(1727).Command-Line Format --innodb_lock_wait_timeout=#
Option-File Format innodb_lock_wait_timeout
Option Sets Variable Yes, innodb_lock_wait_timeout
Variable Name innodb_lock_wait_timeout
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 50
Range 1 .. 1073741824
The timeout in seconds an
InnoDB
transaction waits for a row lock before giving up. The default value is 50 seconds. A transaction that tries to access a row that is locked by anotherInnoDB
transaction waits at most this many seconds for write access to the row before issuing the following error:ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
When a lock wait timeout occurs, the current statement is rolled back (not the entire transaction). To have the entire transaction roll back, start the server with the
--innodb_rollback_on_timeout
option. See also Section 13.3.13, “InnoDB
Error Handling”.You might decrease this value for highly interactive applications or OLTP systems, to display user feedback quickly or put the update into a queue for processing later. You might increase this value for long-running back-end operations, such as a transform step in a data warehouse that waits for other large insert or update operations to finish.
innodb_lock_wait_timeout
applies toInnoDB
row locks only. A MySQL table lock does not happen insideInnoDB
and this timeout does not apply to waits for table locks.The lock wait timeout value does not apply to deadlocks, because
InnoDB
detects them immediately and rolls back one of the deadlocked transactions.innodb_locks_unsafe_for_binlog
Command-Line Format --innodb_locks_unsafe_for_binlog
Option-File Format innodb_locks_unsafe_for_binlog
Option Sets Variable Yes, innodb_locks_unsafe_for_binlog
Variable Name innodb_locks_unsafe_for_binlog
Variable Scope Global Dynamic Variable No Deprecated 5.6.3 Permitted Values Type boolean
Default OFF
This variable affects how
InnoDB
uses gap locking for searches and index scans. Normally,InnoDB
uses an algorithm called next-key locking that combines index-row locking with gap locking.InnoDB
performs row-level locking in such a way that when it searches or scans a table index, it sets shared or exclusive locks on the index records it encounters. Thus, the row-level locks are actually index-record locks. In addition, a next-key lock on an index record also affects the “gap” before that index record. That is, a next-key lock is an index-record lock plus a gap lock on the gap preceding the index record. If one session has a shared or exclusive lock on recordR
in an index, another session cannot insert a new index record in the gap immediately beforeR
in the index order. See Section 13.3.9.4, “InnoDB
Record, Gap, and Next-Key Locks”.By default, the value of
innodb_locks_unsafe_for_binlog
is 0 (disabled), which means that gap locking is enabled:InnoDB
uses next-key locks for searches and index scans. To enable the variable, set it to 1. This causes gap locking to be disabled:InnoDB
uses only index-record locks for searches and index scans.Enabling
innodb_locks_unsafe_for_binlog
does not disable the use of gap locking for foreign-key constraint checking or duplicate-key checking.The effect of enabling
innodb_locks_unsafe_for_binlog
is similar to but not identical to setting the transaction isolation level toREAD COMMITTED
:Enabling
innodb_locks_unsafe_for_binlog
is a global setting and affects all sessions, whereas the isolation level can be set globally for all sessions, or individually per session.innodb_locks_unsafe_for_binlog
can be set only at server startup, whereas the isolation level can be set at startup or changed at runtime.
READ COMMITTED
therefore offers finer and more flexible control thaninnodb_locks_unsafe_for_binlog
. For additional details about the effect of isolation level on gap locking, see Section 12.3.6, “SET TRANSACTION
Синтаксис”.Enabling
innodb_locks_unsafe_for_binlog
may cause phantom problems because other sessions can insert new rows into the gaps when gap locking is disabled. Suppose that there is an index on theid
column of thechild
table and that you want to read and lock all rows from the table having an identifier value larger than 100, with the intention of updating some column in the selected rows later:SELECT * FROM child WHERE id > 100 FOR UPDATE;
The query scans the index starting from the first record where
id
is greater than 100. If the locks set on the index records in that range do not lock out inserts made in the gaps, another session can insert a new row into the table. Consequently, if you were to execute the sameSELECT
again within the same transaction, you would see a new row in the result set returned by the query. This also means that if new items are added to the database,InnoDB
does not guarantee serializability. Therefore, ifinnodb_locks_unsafe_for_binlog
is enabled,InnoDB
guarantees at most an isolation level ofREAD COMMITTED
. (Conflict serializability is still guaranteed.) For additional information about phantoms, see Section 13.3.9.5, “Avoiding the Phantom Problem Using Next-Key Locking”.Enabling
innodb_locks_unsafe_for_binlog
has additional effects:For
UPDATE
orDELETE
statements,InnoDB
holds locks only for rows that it updates or deletes. Record locks for nonmatching rows are released after MySQL has evaluated theWHERE
condition. This greatly reduces the probability of deadlocks, but they can still happen.For
UPDATE
statements, if a row is already locked,InnoDB
performs a “semi-consistent” read, returning the latest committed version to MySQL so that MySQL can determine whether the row matches theWHERE
condition of theUPDATE
. If the row matches (must be updated), MySQL reads the row again and this timeInnoDB
either locks it or waits for a lock on it.
Consider the following example, beginning with this table:
CREATE TABLE t (a INT NOT NULL, b INT) ENGINE = InnoDB; INSERT INTO t VALUES (1,2),(2,3),(3,2),(4,3),(5,2); COMMIT;
In this case, table has no indexes, so searches and index scans use the hidden clustered index for record locking (see Section 13.3.11.1, “Clustered and Secondary Indexes”).
Suppose that one client performs an
UPDATE
using these statements:SET autocommit = 0; UPDATE t SET b = 5 WHERE b = 3;
Suppose also that a second client performs an
UPDATE
by executing these statements following those of the first client:SET autocommit = 0; UPDATE t SET b = 4 WHERE b = 2;
As
InnoDB
executes eachUPDATE
, it first acquires an exclusive lock for each row, and then determines whether to modify it. IfInnoDB
does not modify the row andinnodb_locks_unsafe_for_binlog
is enabled, it releases the lock. Otherwise,InnoDB
retains the lock until the end of the transaction. This affects transaction processing as follows.If
innodb_locks_unsafe_for_binlog
is disabled, the firstUPDATE
acquires x-locks and does not release any of them:x-lock(1,2); retain x-lock x-lock(2,3); update(2,3) to (2,5); retain x-lock x-lock(3,2); retain x-lock x-lock(4,3); update(4,3) to (4,5); retain x-lock x-lock(5,2); retain x-lock
The second
UPDATE
blocks as soon as it tries to acquire any locks (because first update has retained locks on all rows), and does not proceed until the firstUPDATE
commits or rolls back:x-lock(1,2); block and wait for first UPDATE to commit or roll back
If
innodb_locks_unsafe_for_binlog
is enabled, the firstUPDATE
acquires x-locks and releases those for rows that it does not modify:x-lock(1,2); unlock(1,2) x-lock(2,3); update(2,3) to (2,5); retain x-lock x-lock(3,2); unlock(3,2) x-lock(4,3); update(4,3) to (4,5); retain x-lock x-lock(5,2); unlock(5,2)
For the second
UPDATE
,InnoDB
does a “semi-consistent” read, returning the latest committed version of each row to MySQL so that MySQL can determine whether the row matches theWHERE
condition of theUPDATE
:x-lock(1,2); update(1,2) to (1,4); retain x-lock x-lock(2,3); unlock(2,3) x-lock(3,2); update(3,2) to (3,4); retain x-lock x-lock(4,3); unlock(4,3) x-lock(5,2); update(5,2) to (5,4); retain x-lock
Command-Line Format --innodb_log_buffer_size=#
Option-File Format innodb_log_buffer_size
Option Sets Variable Yes, innodb_log_buffer_size
Variable Name innodb_log_buffer_size
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 8388608
Range 262144 .. 4294967295
The size in bytes of the buffer that
InnoDB
uses to write to the log files on disk. The default value is 8MB. A large log buffer enables large transactions to run without a need to write the log to disk before the transactions commit. Thus, if you have big transactions, making the log buffer larger saves disk I/O.Command-Line Format --innodb_log_file_size=#
Option-File Format innodb_log_file_size
Option Sets Variable Yes, innodb_log_file_size
Variable Name innodb_log_file_size
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 5242880
Range 108576 .. 4294967295
The size in bytes of each log file in a log group. The combined size of log files must be less than 4GB. The default value is 5MB. Sensible values range from 1MB to 1/
N
-th of the size of the buffer pool, whereN
is the number of log files in the group. The larger the value, the less checkpoint flush activity is needed in the buffer pool, saving disk I/O. But larger log files also mean that recovery is slower in case of a crash.Command-Line Format --innodb_log_files_in_group=#
Option-File Format innodb_log_files_in_group
Option Sets Variable Yes, innodb_log_files_in_group
Variable Name innodb_log_files_in_group
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 2
Range 2 .. 100
The number of log files in the log group.
InnoDB
writes to the files in a circular fashion. The default (and recommended) value is 2.Command-Line Format --innodb_log_group_home_dir=path
Option-File Format innodb_log_group_home_dir
Option Sets Variable Yes, innodb_log_group_home_dir
Variable Name innodb_log_group_home_dir
Variable Scope Global Dynamic Variable No Permitted Values Type file name
The directory path to the
InnoDB
redo log files. If you do not specify anyInnoDB
log variables, the default is to create two 5MB files namedib_logfile0
andib_logfile1
in the MySQL data directory.Command-Line Format --innodb_max_dirty_pages_pct=#
Option-File Format innodb_max_dirty_pages_pct
Option Sets Variable Yes, innodb_max_dirty_pages_pct
Variable Name innodb_max_dirty_pages_pct
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 75
Range 0 .. 99
This is an integer in the range from 0 to 99. The default value is 75. The main thread in
InnoDB
tries to write pages from the buffer pool so that the percentage of dirty (not yet written) pages will not exceed this value.Command-Line Format --innodb_max_purge_lag=#
Option-File Format innodb_max_purge_lag
Option Sets Variable Yes, innodb_max_purge_lag
Variable Name innodb_max_purge_lag
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
Range 0 .. 4294967295
This variable controls how to delay
INSERT
,UPDATE
, andDELETE
operations when purge operations are lagging (see Section 13.3.10, “InnoDB
Multi-Versioning”). The default value 0 (no delays).The
InnoDB
transaction system maintains a list of transactions that have index records delete-marked byUPDATE
orDELETE
operations. Let the length of this list bepurge_lag
. Whenpurge_lag
exceedsinnodb_max_purge_lag
, eachINSERT
,UPDATE
, andDELETE
operation is delayed by ((purge_lag
/innodb_max_purge_lag
)×10)–5 milliseconds. The delay is computed in the beginning of a purge batch, every ten seconds. The operations are not delayed if purge cannot run because of an old consistent read view that could see the rows to be purged.A typical setting for a problematic workload might be 1 million, assuming that transactions are small, only 100 bytes in size, and it is permissible to have 100MB of unpurged
InnoDB
table rows.The lag value is displayed as the history list length in the
TRANSACTIONS
section of InnoDB Monitor output. For example, if the output includes the following lines, the lag value is 20:------------ TRANSACTIONS ------------ Trx id counter 0 290328385 Purge done for trx's n:o < 0 290315608 undo n:o < 0 17 History list length 20
The number of identical copies of log groups to keep for the database. This should be set to 1.
Command-Line Format --innodb_old_blocks_pct=#
Option-File Format innodb_old_blocks_pct
Variable Name innodb_old_blocks_pct
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 37
Range 5 .. 95
Specifies the approximate percentage of the
InnoDB
buffer pool used for the old block sublist. The range of values is 5 to 95. The default value is 37 (that is, 3/8 of the pool). See Section 7.9.1, “TheInnoDB
Buffer Pool”Command-Line Format --innodb_old_blocks_time=#
Option-File Format innodb_old_blocks_time
Variable Name innodb_old_blocks_time
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
Range 0 .. 2**32-1
Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before it can be moved to the new sublist. The default value is 0: A block inserted into the old sublist moves immediately to the new sublist the first time it is accessed, no matter how soon after insertion the access occurs. If the value is greater than 0, blocks remain in the old sublist until an access occurs at least that many ms after the first access. For example, a value of 1000 causes blocks to stay in the old sublist for 1 second after the first access before they become eligible to move to the new sublist. See Section 7.9.1, “The
InnoDB
Buffer Pool”Command-Line Format --innodb_open_files=#
Option-File Format innodb_open_files
Variable Name innodb_open_files
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 300
Range 10 .. 4294967295
This variable is relevant only if you use multiple tablespaces in
InnoDB
. It specifies the maximum number of.ibd
files thatInnoDB
can keep open at one time. The minimum value is 10. The default value is 300.The file descriptors used for
.ibd
files are forInnoDB
only. They are independent of those specified by the--open-files-limit
server option, and do not affect the operation of the table cache.Version Introduced 5.5.4 Command-Line Format --innodb_purge_batch_size=#
Option-File Format innodb_purge_batch_size
Variable Name innodb_purge_batch_size
Variable Scope Global Dynamic Variable No Permitted Values (>= 5.5.4) Type numeric
Default 20
Range 1 .. 5000
The granularity of changes, expressed in units of redo log records, that trigger a purge operation, flushing the changed buffer pool blocks to disk. The default value is 20, and the range is 1-5000. This option is intended for tuning performance in combination with the setting
innodb_purge_threads=1
, and typical users do not need to modify it.Version Introduced 5.5.4 Command-Line Format --innodb_purge_threads=#
Option-File Format innodb_purge_threads
Variable Name innodb_purge_threads
Variable Scope Global Dynamic Variable No Permitted Values (>= 5.5.4) Type numeric
Default 0
Range 0 .. 1
The number of background threads devoted to the InnoDB purge operation. Currently, can only be 0 (the default) or 1. The default value of 0 signifies that the purge operation is performed as part of the master thread. Running the purge operation in its own thread can reduce internal contention within InnoDB, improving scalability. Currently, the performance gain might be minimal because the background thread might encounter different kinds of contention than before. This feature primarily lays the groundwork for future performance work.
Command-Line Format --innodb_read_ahead_threshold=#
Option-File Format innodb_read_ahead_threshold
Option Sets Variable Yes, innodb_read_ahead_threshold
Variable Name innodb_read_ahead_threshold
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 56
Range 0 .. 64
Controls the sensitivity of linear read-ahead that
InnoDB
uses to prefetch pages into the buffer cache. IfInnoDB
reads at leastinnodb_read_ahead_threshold
pages sequentially from an extent (64 pages), it initiates an asynchronous read for the entire following extent. The permissible range of values is 0 to 64. The default is 56:InnoDB
must read at least 56 pages sequentially from an extent to initiate an asynchronous read for the following extent.Command-Line Format --innodb_read_io_threads=#
Option-File Format innodb_read_io_threads
Option Sets Variable Yes, innodb_read_io_threads
Variable Name innodb_read_io_threads
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 4
Range 1 .. 64
The number of I/O threads for read operations in
InnoDB
. The default value is 4.Command-Line Format --innodb_replication_delay=#
Option-File Format innodb_replication_delay
Option Sets Variable Yes, innodb_replication_delay
Variable Name innodb_replication_delay
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
Range 0 .. 4294967295
The replication thread delay (in ms) on a slave server if
innodb_thread_concurrency
is reached.Command-Line Format --innodb_rollback_on_timeout
Option-File Format innodb_rollback_on_timeout
Option Sets Variable Yes, innodb_rollback_on_timeout
Variable Name innodb_rollback_on_timeout
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
In MySQL 5.5,
InnoDB
rolls back only the last statement on a transaction timeout by default. If--innodb_rollback_on_timeout
is specified, a transaction timeout causesInnoDB
to abort and roll back the entire transaction (the same behavior as in MySQL 4.1).Version Introduced 5.5.11 Command-Line Format --innodb_rollback_segments=#
Option-File Format innodb_rollback_segments
Option Sets Variable Yes, innodb_rollback_segments
Variable Name innodb_rollback_segments
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 128
Range 1 .. 128
Defines how many of the rollback segments in the system tablespace that InnoDB uses within a transaction. You might reduce this value from its default of 128 if a smaller number of rollback segments performs better for your workload.
Command-Line Format --innodb_spin_wait_delay=#
Option-File Format innodb_spin_wait_delay
Option Sets Variable Yes, innodb_spin_wait_delay
Variable Name innodb_spin_wait_delay
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 6
Range 0 .. 4294967295
The maximum delay between polls for a spin lock. The default value is 6.
Version Introduced 5.5.10 Command-Line Format --innodb_stats_method=name
Option-File Format innodb_stats_method
Option Sets Variable Yes, innodb_stats_method
Variable Name innodb_stats_method
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type enumeration
Default nulls_equal
How the server treats
NULL
values when collecting statistics about the distribution of index values forInnoDB
tables. This variable has three possible values,nulls_equal
,nulls_unequal
, andnulls_ignored
. Fornulls_equal
, allNULL
index values are considered equal and form a single value group that has a size equal to the number ofNULL
values. Fornulls_unequal
,NULL
values are considered unequal, and eachNULL
forms a distinct value group of size 1. Fornulls_ignored
,NULL
values are ignored.The method that is used for generating table statistics influences how the optimizer chooses indexes for query execution, as described in Section 7.3.7, “
InnoDB
andMyISAM
Index Statistics Collection”.Version Introduced 5.5.4 Command-Line Format --innodb_stats_on_metadata
Option-File Format innodb_stats_on_metadata
Option Sets Variable Yes, innodb_stats_on_metadata
Variable Name innodb_stats_on_metadata
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default ON
When this variable is enabled (which is the default, as before the variable was created),
InnoDB
updates statistics during metadata statements such asSHOW TABLE STATUS
orSHOW INDEX
, or when accessing theINFORMATION_SCHEMA
tablesTABLES
orSTATISTICS
. (These updates are similar to what happens forANALYZE TABLE
.) When disabled,InnoDB
does not update statistics during these operations. Disabling this variable can improve access speed for schemas that have a large number of tables or indexes. It can also improve the stability of execution plans for queries that involveInnoDB
tables.Command-Line Format --innodb_stats_sample_pages=#
Option-File Format innodb_stats_sample_pages
Option Sets Variable Yes, innodb_stats_sample_pages
Variable Name innodb_stats_sample_pages
Variable Scope Global Dynamic Variable Yes Deprecated 5.6.3 Permitted Values Type numeric
Default 8
Range 1 .. 2**64-1
The number of index pages to sample for index distribution statistics such as are calculated by
ANALYZE TABLE
. The default value is 8. For more information, see Section 13.4.8, “Changes for Flexibility, Ease of Use and Reliability”.Command-Line Format --innodb_strict_mode=#
Option-File Format innodb_strict_mode
Option Sets Variable Yes, innodb_strict_mode
Variable Name innodb_strict_mode
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default OFF
Whether
InnoDB
returns errors rather than warnings for certain conditions. This is analogous to strict SQL mode. The default value isOFF
. See Section 13.4.8.4, “InnoDB Strict Mode” for a list of the conditions that are affected.Command-Line Format --innodb_support_xa
Option-File Format innodb_support_xa
Option Sets Variable Yes, innodb_support_xa
Variable Name innodb_support_xa
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default TRUE
Enables
InnoDB
support for two-phase commit in XA transactions, causing an extra disk flush for transaction preparation. This setting is the default. The XA mechanism is used internally and is essential for any server that has its binary log turned on and is accepting changes to its data from more than one thread. If you turn it off, transactions can be written to the binary log in a different order from the one in which the live database is committing them. This can produce different data when the binary log is replayed in disaster recovery or on a replication slave. Do not turn it off on a replication master server unless you have an unusual setup where only one thread is able to change data.For a server that is accepting data changes from only one thread, it is safe and recommended to turn off this option to improve performance for
InnoDB
tables. For example, you can turn it off on replication slaves where only the replication SQL thread is changing data.You can also turn off this option if you do not need it for safe binary logging or replication, and you also do not use an external XA transaction manager.
Command-Line Format --innodb_sync_spin_loops=#
Option-File Format innodb_sync_spin_loops
Option Sets Variable Yes, innodb_sync_spin_loops
Variable Name innodb_sync_spin_loops
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 30
Range 0 .. 4294967295
The number of times a thread waits for an
InnoDB
mutex to be freed before the thread is suspended. The default value is 30.Command-Line Format --innodb_table_locks
Option-File Format innodb_table_locks
Option Sets Variable Yes, innodb_table_locks
Variable Name innodb_table_locks
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default TRUE
If
autocommit = 0
,InnoDB
honorsLOCK TABLES
; MySQL does not return fromLOCK TABLES ... WRITE
until all other threads have released all their locks to the table. The default value ofinnodb_table_locks
is 1, which means thatLOCK TABLES
causes InnoDB to lock a table internally ifautocommit = 0
.As of MySQL 5.5.3,
innodb_table_locks = 0
has no effect for tables locked explicitly withLOCK TABLES ... WRITE
. It still has an effect for tables locked for read or write byLOCK TABLES ... WRITE
implicitly (for example, through triggers) or byLOCK TABLES ... READ
.Command-Line Format --innodb_thread_concurrency=#
Option-File Format innodb_thread_concurrency
Option Sets Variable Yes, innodb_thread_concurrency
Variable Name innodb_thread_concurrency
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
Range 0 .. 1000
InnoDB
tries to keep the number of operating system threads concurrently insideInnoDB
less than or equal to the limit given by this variable. Once the number of threads reaches this limit, additional threads are placed into a wait state within a FIFO queue for execution. Threads waiting for locks are not counted in the number of concurrently executing threads.The correct value for this variable is dependent on environment and workload. Try a range of different values to determine what value works for your applications. A recommended value is 2 times the number of CPUs plus the number of disks.
The range of this variable is 0 to 1000. A value of 0 (the default) is interpreted as infinite concurrency (no concurrency checking). Disabling thread concurrency checking enables InnoDB to create as many threads as it needs.
Command-Line Format --innodb_thread_sleep_delay=#
Option-File Format innodb_thread_sleep_delay
Option Sets Variable Yes, innodb_thread_sleep_delay
Variable Name innodb_thread_sleep_delay
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 10000
How long
InnoDB
threads sleep before joining theInnoDB
queue, in microseconds. The default value is 10,000. A value of 0 disables sleep.Version Introduced 5.5.4 Command-Line Format --innodb_use_native_aio=#
Option-File Format innodb_use_native_aio
Option Sets Variable Yes, innodb_use_native_aio
Variable Name innodb_use_native_aio
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default ON
Specifies whether to use the Linux asynchronous I/O subsystem. This variable applies to Linux systems only, and cannot be changed while the server is running.
Normally, you do not need to touch this option, because it is enabled by default. If a problem with the asynchronous I/O subsystem in the OS prevents
InnoDB
from starting, start the server with this variable disabled (useinnodb_use_native_aio=0
in the option file). This option could also be turned off automatically during startup, ifInnoDB
detects a potential problem such as a combination oftmpdir
location,tmpfs
filesystem, and Linux kernel that that does not support AIO ontmpfs
.This variable was added in MySQL 5.5.4.
Command-Line Format --innodb_use_sys_malloc=#
Option-File Format innodb_use_sys_malloc
Option Sets Variable Yes, innodb_use_sys_malloc
Variable Name innodb_use_sys_malloc
Variable Scope Global Dynamic Variable No Deprecated 5.6.3 Permitted Values Type boolean
Default ON
Whether
InnoDB
uses the operating system memory allocator (ON
) or its own (OFF
). The default value isON
.The
InnoDB
version number.Command-Line Format --innodb_write_io_threads=#
Option-File Format innodb_write_io_threads
Option Sets Variable Yes, innodb_write_io_threads
Variable Name innodb_write_io_threads
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 4
Range 1 .. 64
The number of I/O threads for write operations in
InnoDB
. The default value is 4.Command-Line Format --sync-binlog=#
Option-File Format sync_binlog
Option Sets Variable Yes, sync_binlog
Variable Name sync_binlog
Variable Scope Global Dynamic Variable Yes Permitted Values Platform Bit Size 32
Type numeric
Default 0
Range 0 .. 4294967295
Permitted Values Platform Bit Size 64
Type numeric
Default 0
Range 0 .. 18446744073709547520
If the value of this variable is greater than 0, the MySQL server synchronizes its binary log to disk (using
fdatasync()
) after everysync_binlog
writes to the binary log. There is one write to the binary log per statement if autocommit is enabled, and one write per transaction otherwise. The default value ofsync_binlog
is 0, which does no synchronizing to disk. A value of 1 is the safest choice, because in the event of a crash you lose at most one statement or transaction from the binary log. However, it is also the slowest choice (unless the disk has a battery-backed cache, which makes synchronization very fast).
To create an InnoDB
table, specify an
ENGINE=InnoDB
option in the
CREATE TABLE
statement:
CREATE TABLE customers (a INT, b CHAR (20), INDEX (a)) ENGINE=InnoDB;
The statement creates a table and an index on column
a
in the InnoDB
tablespace
that consists of the data files that you specified in
my.cnf
. In addition, MySQL creates a file
customers.frm
in the
test
directory under the MySQL database
directory. Internally, InnoDB
adds an entry for
the table to its own data dictionary. The entry includes the
database name. For example, if test
is the
database in which the customers
table is
created, the entry is for 'test/customers'
.
This means you can create a table of the same name
customers
in some other database, and the table
names do not collide inside InnoDB
.
You can query the amount of free space in the
InnoDB
tablespace by issuing a
SHOW TABLE STATUS
statement for any
InnoDB
table. The amount of free space in the
tablespace appears in the Data_free
section in
the output of SHOW TABLE STATUS
.
For example:
SHOW TABLE STATUS FROM test LIKE 'customers'
The statistics SHOW
displays for
InnoDB
tables are only approximate. They are
used in SQL optimization. Table and index reserved sizes in bytes
are accurate, though.
Transactions in SQL
By default, each client that connects to the MySQL server begins
with autocommit mode
enabled, which automatically commits every SQL statement as you
execute it. To use multiple-statement transactions, you can
switch autocommit off with the SQL statement SET
autocommit = 0
and end each transaction with either
COMMIT
or
ROLLBACK
. If
you want to leave autocommit on, you can begin your transactions
within START
TRANSACTION
and end them with
COMMIT
or
ROLLBACK
. The
following example shows two transactions. The first is
committed; the second is rolled back.
shell>mysql test
mysql>CREATE TABLE customer (a INT, b CHAR (20), INDEX (a))
->ENGINE=InnoDB;
Query OK, 0 rows affected (0.00 sec) mysql>START TRANSACTION;
Query OK, 0 rows affected (0.00 sec) mysql>INSERT INTO customer VALUES (10, 'Heikki');
Query OK, 1 row affected (0.00 sec) mysql>COMMIT;
Query OK, 0 rows affected (0.00 sec) mysql>SET autocommit=0;
Query OK, 0 rows affected (0.00 sec) mysql>INSERT INTO customer VALUES (15, 'John');
Query OK, 1 row affected (0.00 sec) mysql>ROLLBACK;
Query OK, 0 rows affected (0.00 sec) mysql>SELECT * FROM customer;
+------+--------+ | a | b | +------+--------+ | 10 | Heikki | +------+--------+ 1 row in set (0.00 sec) mysql>
Transactions in Client-Side Languages
In APIs such as PHP, Perl DBI, JDBC, ODBC, or the standard C
call interface of MySQL, you can send transaction control
statements such as COMMIT
to the
MySQL server as strings just like any other SQL statements such
as SELECT
or
INSERT
. Some APIs also offer
separate special transaction commit and rollback functions or
methods.
To convert a non-InnoDB
table to use
InnoDB
use ALTER
TABLE
:
ALTER TABLE table_name
ENGINE=InnoDB;
Do not convert MySQL system tables in the
mysql
database (such as
user
or host
) to the
InnoDB
type. This is an unsupported
operation. The system tables must always be of the
MyISAM
type.
To make an InnoDB table that is a clone of a MyISAM table:
Create an empty
InnoDB
table with identical definitions.Create the appropriate indexes.
Insert the rows with
INSERT INTO
.innodb_table
SELECT * FROMmyisam_table
You can also create the indexes after inserting the data. Historically, creating new secondary indexes was a slow operation for InnoDB, but this is no longer the case.
If you have UNIQUE
constraints on secondary
keys, you can speed up a table import by turning off the
uniqueness checks temporarily during the import operation:
SET unique_checks=0;... import operation ...
SET unique_checks=1;
For big tables, this saves disk I/O because
InnoDB
can use its
insert buffer to write
secondary index records as a batch. Be certain that the data
contains no duplicate keys.
unique_checks
permits but does
not require storage engines to ignore duplicate keys.
To get better control over the insertion process, you might insert big tables in pieces:
INSERT INTO newtable SELECT * FROM oldtable WHERE yourkey >something
AND yourkey <=somethingelse
;
After all records have been inserted, you can rename the tables.
During the conversion of big tables, increase the size of the
InnoDB
buffer pool to reduce disk I/O, to a
maximum of 80% of physical memory. You can also increase the
sizes of the InnoDB
log files.
Make sure that you do not fill up the tablespace:
InnoDB
tables require a lot more disk space
than MyISAM
tables. If an
ALTER TABLE
operation runs out of
space, it starts a rollback, and that can take hours if it is
disk-bound. For inserts, InnoDB
uses the
insert buffer to merge secondary index records to indexes in
batches. That saves a lot of disk I/O. For rollback, no such
mechanism is used, and the rollback can take 30 times longer
than the insertion.
In the case of a runaway rollback, if you do not have valuable
data in your database, it may be advisable to kill the database
process rather than wait for millions of disk I/O operations to
complete. For the complete procedure, see
Section 13.3.7.2, “Forcing InnoDB
Recovery”.
If you want all new user-created tables to use the
InnoDB
storage engine, add the line
default-storage-engine=innodb
to the
[mysqld]
section of your server option file.
InnoDB
provides an optimization that
significantly improves scalability and performance of SQL
statements that insert rows into tables with
AUTO_INCREMENT
columns. This section provides
background information on the original
(“traditional”) implementation of auto-increment
locking in InnoDB
, explains the configurable
locking mechanism, documents the parameter for configuring the
mechanism, and describes its behavior and interaction with
replication.
The original implementation of auto-increment handling in
InnoDB
uses the following strategy to
prevent problems when using the binary log for statement-based
replication or for certain recovery scenarios.
If you specify an AUTO_INCREMENT
column for
an InnoDB
table, the table handle in the
InnoDB
data dictionary contains a special
counter called the auto-increment counter that is used in
assigning new values for the column. This counter is stored
only in main memory, not on disk.
InnoDB
uses the following algorithm to
initialize the auto-increment counter for a table
t
that contains an
AUTO_INCREMENT
column named
ai_col
: After a server startup, for the
first insert into a table t
,
InnoDB
executes the equivalent of this
statement:
SELECT MAX(ai_col) FROM t FOR UPDATE;
InnoDB
increments the value retrieved by
the statement and assigns it to the column and to the
auto-increment counter for the table. By default, the value is
incremented by one. This default can be overridden by the
auto_increment_increment
configuration setting.
If the table is empty, InnoDB
uses the
value 1
. This default can be overridden by
the auto_increment_offset
configuration setting.
If a SHOW TABLE STATUS
statement examines the table t
before the
auto-increment counter is initialized,
InnoDB
initializes but does not increment
the value and stores it for use by later inserts. This
initialization uses a normal exclusive-locking read on the
table and the lock lasts to the end of the transaction.
InnoDB
follows the same procedure for
initializing the auto-increment counter for a freshly created
table.
After the auto-increment counter has been initialized, if a
you do not explicitly specify a value for an
AUTO_INCREMENT
column,
InnoDB
increments the counter and assigns
the new value to the column. If you insert a row that
explicitly specifies the column value, and the value is bigger
than the current counter value, the counter is set to the
specified column value.
When accessing the auto-increment counter,
InnoDB
uses a special table-level
AUTO-INC
lock that it keeps to the end of
the current SQL statement, not to the end of the transaction.
The special lock release strategy was introduced to improve
concurrency for inserts into a table containing an
AUTO_INCREMENT
column. Nevertheless, two
transactions cannot have the AUTO-INC
lock
on the same table simultaneously, which can have a performance
impact if the AUTO-INC
lock is held for a
long time. That might be the case for a statement such as
INSERT INTO t1 ... SELECT ... FROM t2
that
inserts all rows from one table into another.
InnoDB
uses the in-memory auto-increment
counter as long as the server runs. When the server is stopped
and restarted, InnoDB
reinitializes the
counter for each table for the first
INSERT
to the table, as
described earlier.
You may see gaps in the sequence of values assigned to the
AUTO_INCREMENT
column if you roll back
transactions that have generated numbers using the counter.
If a user specifies NULL
or
0
for the AUTO_INCREMENT
column in an INSERT
,
InnoDB
treats the row as if the value was
not specified and generates a new value for it.
The behavior of the auto-increment mechanism is not defined if you assign a negative value to the column, or if the value becomes bigger than the maximum integer that can be stored in the specified integer type.
An AUTO_INCREMENT
column must appear as the
first column in an index on an InnoDB
table.
InnoDB
supports the AUTO_INCREMENT
=
table option in
N
CREATE TABLE
and
ALTER TABLE
statements, to set
the initial counter value or alter the current counter value.
The effect of this option is canceled by a server restart, for
reasons discussed earlier in this section.
As described in the previous section,
InnoDB
uses a special lock called the
table-level AUTO-INC
lock for inserts into
tables with AUTO_INCREMENT
columns. This
lock is normally held to the end of the statement (not to the
end of the transaction), to ensure that auto-increment numbers
are assigned in a predictable and repeatable order for a given
sequence of INSERT
statements.
In the case of statement-based replication, this means that
when an SQL statement is replicated on a slave server, the
same values are used for the auto-increment column as on the
master server. The result of execution of multiple
INSERT
statements is
deterministic, and the slave reproduces the same data as on
the master. If auto-increment values generated by multiple
INSERT
statements were
interleaved, the result of two concurrent
INSERT
statements would be
nondeterministic, and could not reliably be propagated to a
slave server using statement-based replication.
To make this clear, consider an example that uses this table:
CREATE TABLE t1 ( c1 INT(11) NOT NULL AUTO_INCREMENT, c2 VARCHAR(10) DEFAULT NULL, PRIMARY KEY (c1) ) ENGINE=InnoDB;
Suppose that there are two transactions running, each
inserting rows into a table with an
AUTO_INCREMENT
column. One transaction is
using an INSERT
... SELECT
statement that inserts 1000 rows, and
another is using a simple
INSERT
statement that inserts
one row:
Tx1: INSERT INTO t1 (c2) SELECT 1000 rows from another table ... Tx2: INSERT INTO t1 (c2) VALUES ('xxx');
InnoDB
cannot tell in advance how many rows
will be retrieved from the
SELECT
in the
INSERT
statement in Tx1, and it
assigns the auto-increment values one at a time as the
statement proceeds. With a table-level lock, held to the end
of the statement, only one
INSERT
statement referring to
table t1
can execute at a time, and the
generation of auto-increment numbers by different statements
is not interleaved. The auto-increment value generated by the
Tx1 INSERT ...
SELECT
statement will be consecutive, and the
(single) auto-increment value used by the
INSERT
statement in Tx2 will
either be smaller or larger than all those used for Tx1,
depending on which statement executes first.
As long as the SQL statements execute in the same order when
replayed from the binary log (when using statement-based
replication, or in recovery scenarios), the results will be
the same as they were when Tx1 and Tx2 first ran. Thus,
table-level locks held until the end of a statement make
INSERT
statements using
auto-increment safe for use with statement-based replication.
However, those locks limit concurrency and scalability when
multiple transactions are executing insert statements at the
same time.
In the preceding example, if there were no table-level lock,
the value of the auto-increment column used for the
INSERT
in Tx2 depends on
precisely when the statement executes. If the
INSERT
of Tx2 executes while
the INSERT
of Tx1 is running
(rather than before it starts or after it completes), the
specific auto-increment values assigned by the two
INSERT
statements are
nondeterministic, and may vary from run to run.
InnoDB
can avoid using the table-level
AUTO-INC
lock for a class of
INSERT
statements where the
number of rows is known in advance, and still preserve
deterministic execution and safety for statement-based
replication. Further, if you are not using the binary log to
replay SQL statements as part of recovery or replication, you
can entirely eliminate use of the table-level
AUTO-INC
lock for even greater concurrency
and performance—at the cost of permitting gaps in
auto-increment numbers assigned by a statement and potentially
having the numbers assigned by concurrently executing
statements interleaved.
For INSERT
statements where the
number of rows to be inserted is known at the beginning of
processing the statement, InnoDB
quickly
allocates the required number of auto-increment values without
taking any lock, but only if there is no concurrent session
already holding the table-level AUTO-INC
lock (because that other statement will be allocating
auto-increment values one-by-one as it proceeds). More
precisely, such an INSERT
statement obtains auto-increment values under the control of a
mutex (a light-weight lock) that is not
held until the statement completes, but only for the duration
of the allocation process.
This new locking scheme enables much greater scalability, but
it does introduce some subtle differences in how
auto-increment values are assigned compared to the original
mechanism. To describe the way auto-increment works in
InnoDB
, the following discussion defines
some terms, and explains how InnoDB
behaves
using different settings of the new
innodb_autoinc_lock_mode
configuration parameter. Additional considerations are
described following the explanation of auto-increment locking
behavior.
First, some definitions:
“
INSERT
-like” statementsAll statements that generate new rows in a table, including
INSERT
,INSERT ... SELECT
,REPLACE
,REPLACE ... SELECT
, andLOAD DATA
.“Simple inserts”
Statements for which the number of rows to be inserted can be determined in advance (when the statement is initially processed). This includes single-row and multiple-row
INSERT
andREPLACE
statements that do not have a nested subquery, but notINSERT ... ON DUPLICATE KEY UPDATE
.“Bulk inserts”
Statements for which the number of rows to be inserted (and the number of required auto-increment values) is not known in advance. This includes
INSERT ... SELECT
,REPLACE ... SELECT
, andLOAD DATA
statements, but not plainINSERT
.InnoDB
will assign new values for theAUTO_INCREMENT
column one at a time as each row is processed.“Mixed-mode inserts”
These are “simple insert” statements that specify the auto-increment value for some (but not all) of the new rows. An example follows, where
c1
is anAUTO_INCREMENT
column of tablet1
:INSERT INTO t1 (c1,c2) VALUES (1,'a'), (NULL,'b'), (5,'c'), (NULL,'d');
Another type of “mixed-mode insert” is
INSERT ... ON DUPLICATE KEY UPDATE
, which in the worst case is in effect anINSERT
followed by aUPDATE
, where the allocated value for theAUTO_INCREMENT
column may or may not be used during the update phase.
In MySQL 5.5, there is a configuration parameter
that controls how InnoDB
uses locking when
generating values for AUTO_INCREMENT
columns. This parameter can be set using the
--innodb-autoinc-lock-mode
option at mysqld startup.
In general, if you encounter problems with the way auto-increment works (which will most likely involve replication), you can force use of the original behavior by setting the lock mode to 0.
There are three possible settings for the
innodb_autoinc_lock_mode
parameter:
innodb_autoinc_lock_mode = 0
(“traditional” lock mode)This lock mode provides the same behavior as before
innodb_autoinc_lock_mode
existed. For all “INSERT
-like” statements, a special table-levelAUTO-INC
lock is obtained and held to the end of the statement. This assures that the auto-increment values assigned by any given statement are consecutive (although “gaps” can exist within a table if a transaction that generated auto-increment values is rolled back, as discussed later).This lock mode is provided only for backward compatibility and performance testing. There is little reason to use this lock mode unless you use “mixed-mode inserts” and care about the important difference in semantics described later.
innodb_autoinc_lock_mode = 1
(“consecutive” lock mode)This is the default lock mode. In this mode, “bulk inserts” use the special
AUTO-INC
table-level lock and hold it until the end of the statement. This applies to allINSERT ... SELECT
,REPLACE ... SELECT
, andLOAD DATA
statements. Only one statement holding theAUTO-INC
lock can execute at a time.With this lock mode, “simple inserts” (only) use a new locking model where a light-weight mutex is used during the allocation of auto-increment values, and no table-level
AUTO-INC
lock is used, unless anAUTO-INC
lock is held by another transaction. If another transaction does hold anAUTO-INC
lock, a “simple insert” waits for theAUTO-INC
lock, as if it too were a “bulk insert.”This lock mode ensures that, in the presence of
INSERT
statements where the number of rows is not known in advance (and where auto-increment numbers are assigned as the statement progresses), all auto-increment values assigned by any “INSERT
-like” statement are consecutive, and operations are safe for statement-based replication.Simply put, the important impact of this lock mode is significantly better scalability. This mode is safe for use with statement-based replication. Further, as with “traditional” lock mode, auto-increment numbers assigned by any given statement are consecutive. In this mode, there is no change in semantics compared to “traditional” mode for any statement that uses auto-increment, with one important exception.
The exception is for “mixed-mode inserts”, where the user provides explicit values for an
AUTO_INCREMENT
column for some, but not all, rows in a multiple-row “simple insert.” For such inserts,InnoDB
will allocate more auto-increment values than the number of rows to be inserted. However, all values automatically assigned are consecutively generated (and thus higher than) the auto-increment value generated by the most recently executed previous statement. “Excess” numbers are lost.A similar situation exists if you use
INSERT ... ON DUPLICATE KEY UPDATE
. This statement is also classified as a “mixed-mode insert” since an auto-increment value is not necessarily generated for each row. BecauseInnoDB
allocates the auto-increment value before the insert is actually attempted, it cannot know whether an inserted value will be a duplicate of an existing value and thus cannot know whether the auto-increment value it generates will be used for a new row. Therefore, if you are using statement-based replication, either avoidINSERT ... ON DUPLICATE KEY UPDATE
or useinnodb_autoinc_lock_mode = 0
(“traditional” lock mode).innodb_autoinc_lock_mode = 2
(“interleaved” lock mode)In this lock mode, no “
INSERT
-like” statements use the table-levelAUTO-INC
lock, and multiple statements can execute at the same time. This is the fastest and most scalable lock mode, but it is not safe when using statement-based replication or recovery scenarios when SQL statements are replayed from the binary log.In this lock mode, auto-increment values are guaranteed to be unique and monotonically increasing across all concurrently executing “
INSERT
-like” statements. However, because multiple statements can be generating numbers at the same time (that is, allocation of numbers is interleaved across statements), the values generated for the rows inserted by any given statement may not be consecutive.If the only statements executing are “simple inserts” where the number of rows to be inserted is known ahead of time, there will be no gaps in the numbers generated for a single statement, except for “mixed-mode inserts.” However, when “bulk inserts” are executed, there may be gaps in the auto-increment values assigned by any given statement.
The auto-increment locking modes provided by
innodb_autoinc_lock_mode
have
several usage implications:
Using auto-increment with replication
If you are using statement-based replication, set
innodb_autoinc_lock_mode
to 0 or 1 and use the same value on the master and its slaves. Auto-increment values are not ensured to be the same on the slaves as on the master if you useinnodb_autoinc_lock_mode
= 2 (“interleaved”) or configurations where the master and slaves do not use the same lock mode.If you are using row-based replication, all of the auto-increment lock modes are safe. Row-based replication is not sensitive to the order of execution of the SQL statements.
“Lost” auto-increment values and sequence gaps
In all lock modes (0, 1, and 2), if a transaction that generated auto-increment values rolls back, those auto-increment values are “lost.” Once a value is generated for an auto-increment column, it cannot be rolled back, whether or not the “
INSERT
-like” statement is completed, and whether or not the containing transaction is rolled back. Such lost values are not reused. Thus, there may be gaps in the values stored in anAUTO_INCREMENT
column of a table.Gaps in auto-increment values for “bulk inserts”
With
innodb_autoinc_lock_mode
set to 0 (“traditional”) or 1 (“consecutive”), the auto-increment values generated by any given statement will be consecutive, without gaps, because the table-levelAUTO-INC
lock is held until the end of the statement, and only one such statement can execute at a time.With
innodb_autoinc_lock_mode
set to 2 (“interleaved”), there may be gaps in the auto-increment values generated by “bulk inserts,” but only if there are concurrently executing “INSERT
-like” statements.For lock modes 1 or 2, gaps may occur between successive statements because for bulk inserts the exact number of auto-increment values required by each statement may not be known and overestimation is possible.
Auto-increment values assigned by “mixed-mode inserts”
Consider a “mixed-mode insert,” where a “simple insert” specifies the auto-increment value for some (but not all) resulting rows. Such a statement will behave differently in lock modes 0, 1, and 2. For example, assume
c1
is anAUTO_INCREMENT
column of tablet1
, and that the most recent automatically generated sequence number is 100. Consider the following “mixed-mode insert” statement:INSERT INTO t1 (c1,c2) VALUES (1,'a'), (NULL,'b'), (5,'c'), (NULL,'d');
With
innodb_autoinc_lock_mode
set to 0 (“traditional”), the four new rows will be:+-----+------+ | c1 | c2 | +-----+------+ | 1 | a | | 101 | b | | 5 | c | | 102 | d | +-----+------+
The next available auto-increment value will be 103 because the auto-increment values are allocated one at a time, not all at once at the beginning of statement execution. This result is true whether or not there are concurrently executing “
INSERT
-like” statements (of any type).With
innodb_autoinc_lock_mode
set to 1 (“consecutive”), the four new rows will also be:+-----+------+ | c1 | c2 | +-----+------+ | 1 | a | | 101 | b | | 5 | c | | 102 | d | +-----+------+
However, in this case, the next available auto-increment value will be 105, not 103 because four auto-increment values are allocated at the time the statement is processed, but only two are used. This result is true whether or not there are concurrently executing “
INSERT
-like” statements (of any type).With
innodb_autoinc_lock_mode
set to mode 2 (“interleaved”), the four new rows will be:+-----+------+ | c1 | c2 | +-----+------+ | 1 | a | |
x
| b | | 5 | c | |y
| d | +-----+------+The values of
x
andy
will be unique and larger than any previously generated rows. However, the specific values ofx
andy
will depend on the number of auto-increment values generated by concurrently executing statements.Finally, consider the following statement, issued when the most-recently generated sequence number was the value 4:
INSERT INTO t1 (c1,c2) VALUES (1,'a'), (NULL,'b'), (5,'c'), (NULL,'d');
With any
innodb_autoinc_lock_mode
setting, this statement will generate a duplicate-key error 23000 (Can't write; duplicate key in table
) because 5 will be allocated for the row(NULL, 'b')
and insertion of the row(5, 'c')
will fail.
InnoDB
supports foreign key constraints. The
syntax for a foreign key constraint definition in
InnoDB
looks like this:
[CONSTRAINT [symbol
]] FOREIGN KEY [index_name
] (index_col_name
, ...) REFERENCEStbl_name
(index_col_name
,...) [ON DELETEreference_option
] [ON UPDATEreference_option
]reference_option
: RESTRICT | CASCADE | SET NULL | NO ACTION
index_name
represents a foreign key
ID. If given, this is ignored if an index for the foreign key is
defined explicitly. Otherwise, if InnoDB
creates an index for the foreign key, it uses
index_name
for the index name.
Foreign keys definitions are subject to the following conditions:
Both tables must be
InnoDB
tables and they must not beTEMPORARY
tables.Corresponding columns in the foreign key and the referenced key must have similar internal data types inside
InnoDB
so that they can be compared without a type conversion. The size and sign of integer types must be the same. The length of string types need not be the same. For nonbinary (character) string columns, the character set and collation must be the same.InnoDB
requires indexes on foreign keys and referenced keys so that foreign key checks can be fast and not require a table scan. In the referencing table, there must be an index where the foreign key columns are listed as the first columns in the same order. Such an index is created on the referencing table automatically if it does not exist. This index might be silently dropped later, if you create another index that can be used to enforce the foreign key constraint.index_name
, if given, is used as described previously.InnoDB
permits a foreign key to reference any index column or group of columns. However, in the referenced table, there must be an index where the referenced columns are listed as the first columns in the same order.Index prefixes on foreign key columns are not supported. One consequence of this is that
BLOB
andTEXT
columns cannot be included in a foreign key because indexes on those columns must always include a prefix length.If the
CONSTRAINT
clause is given, thesymbol
symbol
value must be unique in the database. If the clause is not given,InnoDB
creates the name automatically.
InnoDB
rejects any
INSERT
or
UPDATE
operation that attempts to
create a foreign key value in a child table if there is no a
matching candidate key value in the parent table. When an
UPDATE
or
DELETE
operation affects a key
value in the parent table that has matching rows in the child
table, the result depends on the referential
action specified using ON UPDATE
and ON DELETE
subclauses of the
FOREIGN KEY
clause. InnoDB
supports five options regarding the action to be taken. If
ON DELETE
or ON UPDATE
are
not specified, the default action is
RESTRICT
.
CASCADE
: Delete or update the row from the parent table, and automatically delete or update the matching rows in the child table. BothON DELETE CASCADE
andON UPDATE CASCADE
are supported. Between two tables, do not define severalON UPDATE CASCADE
clauses that act on the same column in the parent table or in the child table.ЗамечаниеCurrently, cascaded foreign key actions do not activate triggers.
SET NULL
: Delete or update the row from the parent table, and set the foreign key column or columns in the child table toNULL
. BothON DELETE SET NULL
andON UPDATE SET NULL
clauses are supported.If you specify a
SET NULL
action, make sure that you have not declared the columns in the child table asNOT NULL
.RESTRICT
: Rejects the delete or update operation for the parent table. SpecifyingRESTRICT
(orNO ACTION
) is the same as omitting theON DELETE
orON UPDATE
clause.NO ACTION
: A keyword from standard SQL. In MySQL, equivalent toRESTRICT
.InnoDB
rejects the delete or update operation for the parent table if there is a related foreign key value in the referenced table. Some database systems have deferred checks, andNO ACTION
is a deferred check. In MySQL, foreign key constraints are checked immediately, soNO ACTION
is the same asRESTRICT
.SET DEFAULT
: This action is recognized by the parser, butInnoDB
rejects table definitions containingON DELETE SET DEFAULT
orON UPDATE SET DEFAULT
clauses.
InnoDB
supports foreign key references
between one column and another within a table. (A column cannot
have a foreign key reference to itself.) In these cases,
“child table records” really refers to dependent
records within the same table.
Examples of Foreign Key Clauses
Here is a simple example that relates parent
and child
tables through a single-column
foreign key:
CREATE TABLE parent (id INT NOT NULL, PRIMARY KEY (id) ) ENGINE=INNODB; CREATE TABLE child (id INT, parent_id INT, INDEX par_ind (parent_id), FOREIGN KEY (parent_id) REFERENCES parent(id) ON DELETE CASCADE ) ENGINE=INNODB;
A more complex example in which a
product_order
table has foreign keys for two
other tables. One foreign key references a two-column index in
the product
table. The other references a
single-column index in the customer
table:
CREATE TABLE product (category INT NOT NULL, id INT NOT NULL, price DECIMAL, PRIMARY KEY(category, id)) ENGINE=INNODB; CREATE TABLE customer (id INT NOT NULL, PRIMARY KEY (id)) ENGINE=INNODB; CREATE TABLE product_order (no INT NOT NULL AUTO_INCREMENT, product_category INT NOT NULL, product_id INT NOT NULL, customer_id INT NOT NULL, PRIMARY KEY(no), INDEX (product_category, product_id), FOREIGN KEY (product_category, product_id) REFERENCES product(category, id) ON UPDATE CASCADE ON DELETE RESTRICT, INDEX (customer_id), FOREIGN KEY (customer_id) REFERENCES customer(id)) ENGINE=INNODB;
InnoDB
enables you to add a new foreign key
constraint to a table by using ALTER
TABLE
:
ALTER TABLEtbl_name
ADD [CONSTRAINT [symbol
]] FOREIGN KEY [index_name
] (index_col_name
, ...) REFERENCEStbl_name
(index_col_name
,...) [ON DELETEreference_option
] [ON UPDATEreference_option
]
The foreign key can be self referential (referring to the same
table). When you add a foreign key constraint to a table using
ALTER TABLE
, remember
to create the required indexes first.
Foreign Keys and ALTER TABLE
InnoDB
supports the use of
ALTER TABLE
to drop foreign keys:
ALTER TABLEtbl_name
DROP FOREIGN KEYfk_symbol
;
If the FOREIGN KEY
clause included a
CONSTRAINT
name when you created the foreign
key, you can refer to that name to drop the foreign key.
Otherwise, the fk_symbol
value is
internally generated by InnoDB
when the
foreign key is created. To find out the symbol value when you
want to drop a foreign key, use the SHOW
CREATE TABLE
statement. For example:
mysql>SHOW CREATE TABLE ibtest11c\G
*************************** 1. row *************************** Table: ibtest11c Create Table: CREATE TABLE `ibtest11c` ( `A` int(11) NOT NULL auto_increment, `D` int(11) NOT NULL default '0', `B` varchar(200) NOT NULL default '', `C` varchar(175) default NULL, PRIMARY KEY (`A`,`D`,`B`), KEY `B` (`B`,`C`), KEY `C` (`C`), CONSTRAINT `0_38775` FOREIGN KEY (`A`, `D`) REFERENCES `ibtest11a` (`A`, `D`) ON DELETE CASCADE ON UPDATE CASCADE, CONSTRAINT `0_38776` FOREIGN KEY (`B`, `C`) REFERENCES `ibtest11a` (`B`, `C`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=INNODB CHARSET=latin1 1 row in set (0.01 sec) mysql>ALTER TABLE ibtest11c DROP FOREIGN KEY `0_38775`;
You cannot add a foreign key and drop a foreign key in separate
clauses of a single ALTER TABLE
statement. Separate statements are required.
If ALTER TABLE
for an
InnoDB
table results in changes to column
values (for example, because a column is truncated),
InnoDB
's FOREIGN KEY
constraint checks do not notice possible violations caused by
changing the values.
How Foreign Keys Work with Other MySQL Command
The InnoDB
parser permits table and column
identifiers in a FOREIGN KEY ... REFERENCES
...
clause to be quoted within backticks.
(Alternatively, double quotation marks can be used if the
ANSI_QUOTES
SQL mode is
enabled.) The InnoDB
parser also takes into
account the setting of the
lower_case_table_names
system
variable.
InnoDB
returns a table's foreign key
definitions as part of the output of the
SHOW CREATE TABLE
statement:
SHOW CREATE TABLE tbl_name
;
mysqldump also produces correct definitions of tables in the dump file, and does not forget about the foreign keys.
To make it easier to reload dump files for tables that have
foreign key relationships, mysqldump
automatically includes a statement in the dump output to set
foreign_key_checks
to 0. This
avoids problems with tables having to be reloaded in a
particular order when the dump is reloaded. It is also possible
to set this variable manually:
mysql>SET foreign_key_checks = 0;
mysql>SOURCE
mysql>dump_file_name
;SET foreign_key_checks = 1;
This enables you to import the tables in any order if the dump
file contains tables that are not correctly ordered for foreign
keys. It also speeds up the import operation. Setting
foreign_key_checks
to 0 can
also be useful for ignoring foreign key constraints during
LOAD DATA
and
ALTER TABLE
operations. However,
even if foreign_key_checks = 0
,
InnoDB does not permit the creation of a foreign key constraint
where a column references a nonmatching column type. Also, if an
InnoDB
table has foreign key constraints,
ALTER TABLE
cannot be used to
change the table to use another storage engine. To alter the
storage engine, drop any foreign key constraints first.
InnoDB
does not permit you to drop a table
that is referenced by a FOREIGN KEY
constraint, unless you do SET foreign_key_checks =
0
. When you drop a table, the constraints that were
defined in its create statement are also dropped.
If you re-create a table that was dropped, it must have a definition that conforms to the foreign key constraints referencing it. It must have the right column names and types, and it must have indexes on the referenced keys, as stated earlier. If these are not satisfied, MySQL returns error number 1005 and refers to error 150 in the error message.
If MySQL reports an error number 1005 from a
CREATE TABLE
statement, and the
error message refers to error 150, table creation failed because
a foreign key constraint was not correctly formed. Similarly, if
an ALTER TABLE
fails and it
refers to error 150, that means a foreign key definition would
be incorrectly formed for the altered table. You can use
SHOW ENGINE INNODB
STATUS
to display a detailed explanation of the most
recent InnoDB
foreign key error in the
server.
For users familiar with the ANSI/ISO SQL Standard, please note
that no storage engine, including InnoDB
,
recognizes or enforces the MATCH
clause
used in referential-integrity constraint definitions. Use of
an explicit MATCH
clause will not have the
specified effect, and also causes ON DELETE
and ON UPDATE
clauses to be ignored. For
these reasons, specifying MATCH
should be
avoided.
The MATCH
clause in the SQL standard
controls how NULL
values in a composite
(multiple-column) foreign key are handled when comparing to a
primary key. InnoDB
essentially implements
the semantics defined by MATCH SIMPLE
,
which permit a foreign key to be all or partially
NULL
. In that case, the (child table) row
containing such a foreign key is permitted to be inserted, and
does not match any row in the referenced (parent) table. It is
possible to implement other semantics using triggers.
Additionally, MySQL and InnoDB
require that
the referenced columns be indexed for performance. However,
the system does not enforce a requirement that the referenced
columns be UNIQUE
or be declared
NOT NULL
. The handling of foreign key
references to nonunique keys or keys that contain
NULL
values is not well defined for
operations such as UPDATE
or
DELETE CASCADE
. You are advised to use
foreign keys that reference only UNIQUE
and
NOT NULL
keys.
Furthermore, InnoDB
does not recognize or
support “inline REFERENCES
specifications” (as defined in the SQL standard) where
the references are defined as part of the column
specification. InnoDB
accepts
REFERENCES
clauses only when specified as
part of a separate FOREIGN KEY
specification. For other storage engines, MySQL Server parses
and ignores foreign key specifications.
Deviation from SQL standards:
If there are several rows in the parent table that have the same
referenced key value, InnoDB
acts in foreign
key checks as if the other parent rows with the same key value
do not exist. For example, if you have defined a
RESTRICT
type constraint, and there is a
child row with several parent rows, InnoDB
does not permit the deletion of any of those parent rows.
InnoDB
performs cascading operations through
a depth-first algorithm, based on records in the indexes
corresponding to the foreign key constraints.
Deviation from SQL standards: A
FOREIGN KEY
constraint that references a
non-UNIQUE
key is not standard SQL. It is an
InnoDB
extension to standard SQL.
Deviation from SQL standards:
If ON UPDATE CASCADE
or ON UPDATE
SET NULL
recurses to update the same
table it has previously updated during the cascade,
it acts like RESTRICT
. This means that you
cannot use self-referential ON UPDATE CASCADE
or ON UPDATE SET NULL
operations. This is to
prevent infinite loops resulting from cascaded updates. A
self-referential ON DELETE SET NULL
, on the
other hand, is possible, as is a self-referential ON
DELETE CASCADE
. Cascading operations may not be nested
more than 15 levels deep.
Deviation from SQL standards:
Like MySQL in general, in an SQL statement that inserts,
deletes, or updates many rows, InnoDB
checks
UNIQUE
and FOREIGN KEY
constraints row-by-row. When performing foreign key checks,
InnoDB
sets shared row-level locks on child
or parent records it has to look at. InnoDB
checks foreign key constraints immediately; the check is not
deferred to transaction commit. According to the SQL standard,
the default behavior should be deferred checking. That is,
constraints are only checked after the entire SQL
statement has been processed. Until
InnoDB
implements deferred constraint
checking, some things will be impossible, such as deleting a
record that refers to itself using a foreign key.
MySQL replication works for InnoDB
tables as
it does for MyISAM
tables. It is also
possible to use replication in a way where the storage engine on
the slave is not the same as the original storage engine on the
master. For example, you can replicate modifications to an
InnoDB
table on the master to a
MyISAM
table on the slave.
To set up a new slave for a master, you have to make a copy of
the InnoDB
tablespace and the log files, as
well as the .frm
files of the
InnoDB
tables, and move the copies to the
slave. If the
innodb_file_per_table
variable
is enabled, copy the .ibd
files as well.
For the proper procedure to do this, see
Section 13.3.7, “Backing Up and Recovering an InnoDB
Database”.
If you can shut down the master or an existing slave, you can
take a cold backup of the InnoDB
tablespace
and log files and use that to set up a slave. To make a new
slave without taking down any server you can also use the MySQL
Enterprise Backup product.
Transactions that fail on the master do not affect replication
at all. MySQL replication is based on the binary log where MySQL
writes SQL statements that modify data. A transaction that fails
(for example, because of a foreign key violation, or because it
is rolled back) is not written to the binary log, so it is not
sent to slaves. See Section 12.3.1, “START TRANSACTION
,
COMMIT
, and
ROLLBACK
Синтаксис”.
Replication and CASCADE
.
Cascading actions for InnoDB
tables on the
master are replicated on the slave only
if the tables sharing the foreign key relation use
InnoDB
on both the master and slave. This
is true whether you are using statement-based or row-based
replication. Suppose that you have started replication, and
then create two tables on the master using the following
CREATE TABLE
statements:
CREATE TABLE fc1 ( i INT PRIMARY KEY, j INT ) ENGINE = InnoDB; CREATE TABLE fc2 ( m INT PRIMARY KEY, n INT, FOREIGN KEY ni (n) REFERENCES fc1 (i) ON DELETE CASCADE ) ENGINE = InnoDB;
Suppose that the slave does not have InnoDB
support enabled. If this is the case, then the tables on the
slave are created, but they use the MyISAM
storage engine, and the FOREIGN KEY
option is
ignored. Now we insert some rows into the tables on the master:
master>INSERT INTO fc1 VALUES (1, 1), (2, 2);
Query OK, 2 rows affected (0.09 sec) Records: 2 Duplicates: 0 Warnings: 0 master>INSERT INTO fc2 VALUES (1, 1), (2, 2), (3, 1);
Query OK, 3 rows affected (0.19 sec) Records: 3 Duplicates: 0 Warnings: 0
At this point, on both the master and the slave, table
fc1
contains 2 rows, and table
fc2
contains 3 rows, as shown here:
master>SELECT * FROM fc1;
+---+------+ | i | j | +---+------+ | 1 | 1 | | 2 | 2 | +---+------+ 2 rows in set (0.00 sec) master>SELECT * FROM fc2;
+---+------+ | m | n | +---+------+ | 1 | 1 | | 2 | 2 | | 3 | 1 | +---+------+ 3 rows in set (0.00 sec) slave>SELECT * FROM fc1;
+---+------+ | i | j | +---+------+ | 1 | 1 | | 2 | 2 | +---+------+ 2 rows in set (0.00 sec) slave>SELECT * FROM fc2;
+---+------+ | m | n | +---+------+ | 1 | 1 | | 2 | 2 | | 3 | 1 | +---+------+ 3 rows in set (0.00 sec)
Now suppose that you perform the following
DELETE
statement on the master:
master> DELETE FROM fc1 WHERE i=1;
Query OK, 1 row affected (0.09 sec)
Due to the cascade, table fc2
on the master
now contains only 1 row:
master> SELECT * FROM fc2;
+---+---+
| m | n |
+---+---+
| 2 | 2 |
+---+---+
1 row in set (0.00 sec)
However, the cascade does not propagate on the slave because on
the slave the DELETE
for
fc1
deletes no rows from
fc2
. The slave's copy of
fc2
still contains all of the rows that were
originally inserted:
slave> SELECT * FROM fc2;
+---+---+
| m | n |
+---+---+
| 1 | 1 |
| 3 | 1 |
| 2 | 2 |
+---+---+
3 rows in set (0.00 sec)
This difference is due to the fact that the cascading deletes
are handled internally by the InnoDB
storage
engine, which means that none of the changes are logged.
This section describes what you can do when your
InnoDB
tablespace runs out of room or when you
want to change the size of the log files.
The easiest way to increase the size of the
InnoDB
tablespace is to configure it from the
beginning to be auto-extending. Specify the
autoextend
attribute for the last data file in
the tablespace definition. Then InnoDB
increases the size of that file automatically in 8MB increments
when it runs out of space. The increment size can be changed by
setting the value of the
innodb_autoextend_increment
system variable, which is measured in MB.
Alternatively, you can increase the size of your tablespace by
adding another data file. To do this, you have to shut down the
MySQL server, change the tablespace configuration to add a new
data file to the end of
innodb_data_file_path
, and start
the server again.
If your last data file was defined with the keyword
autoextend
, the procedure for reconfiguring the
tablespace must take into account the size to which the last data
file has grown. Obtain the size of the data file, round it down to
the closest multiple of 1024 × 1024 bytes (= 1MB), and
specify the rounded size explicitly in
innodb_data_file_path
. Then you
can add another data file. Remember that only the last data file
in the innodb_data_file_path
can
be specified as auto-extending.
As an example, assume that the tablespace has just one
auto-extending data file ibdata1
:
innodb_data_home_dir = innodb_data_file_path = /ibdata/ibdata1:10M:autoextend
Suppose that this data file, over time, has grown to 988MB. Here is the configuration line after modifying the original data file to not be auto-extending and adding another auto-extending data file:
innodb_data_home_dir = innodb_data_file_path = /ibdata/ibdata1:988M;/disk2/ibdata2:50M:autoextend
When you add a new file to the tablespace configuration, make sure
that it does not exist. InnoDB
will create and
initialize the file when you restart the server.
Currently, you cannot remove a data file from the tablespace. To decrease the size of your tablespace, use this procedure:
Use mysqldump to dump all your
InnoDB
tables.Stop the server.
Remove all the existing tablespace files, including the
ibdata
andib_log
files. If you want to keep a backup copy of the information, then copy all theib*
files to another location before the removing the files in your MySQL installation.Remove any
.frm
files forInnoDB
tables.Configure a new tablespace.
Restart the server.
Import the dump files.
If you want to change the number or the size of your
InnoDB
log files, use the following
instructions. The procedure to use depends on the value of
innodb_fast_shutdown
:
If
innodb_fast_shutdown
is not set to 2: Stop the MySQL server and make sure that it shuts down without errors (to ensure that there is no information for outstanding transactions in the log). Copy the old log files into a safe place in case something went wrong during the shutdown and you need them to recover the tablespace. Delete the old log files from the log file directory, editmy.cnf
to change the log file configuration, and start the MySQL server again. mysqld sees that noInnoDB
log files exist at startup and creates new ones.If
innodb_fast_shutdown
is set to 2: Setinnodb_fast_shutdown
to 1:mysql>
SET GLOBAL innodb_fast_shutdown = 1;
Then follow the instructions in the previous item.
The key to safe database management is making regular backups. Depending on your data volume, number of MySQL servers, and database workload, you can use these techniques, alone or in combination: hot backup with MySQL Enterprise Backup; cold backup by copying files while the MySQL server is shut down; physical backup for fast operation (especially for restore); logical backup with mysqldump for smaller data volumes or to record the structure of schema objects.
Hot Backups
The mysqlbackup command, part of the MySQL
Enterprise Backup component, lets you back up a running MySQL
instance, including InnoDB
and
MyISAM
tables, with minimal
disruption to operations while producing a consistent snapshot of
the database. When mysqlbackup is copying
InnoDB
tables, reads and writes to both
InnoDB
and MyISAM
tables can continue. During the copying of
MyISAM
tables, reads (but not writes) to those
tables are permitted. MySQL Enterprise Backup can also create
compressed backup files, and back up subsets of tables and
databases. In conjunction with MySQL’s binary log, users can
perform point-in-time recovery. MySQL Enterprise Backup is part of
the MySQL Enterprise subscription. For more details, see
Глава 24, MySQL Enterprise Backup.
Cold Backups
If you can shut down your MySQL server, you can make a binary
backup that consists of all files used by
InnoDB
to manage its tables. Use the
following procedure:
Do a slow shutdown of the MySQL server and make sure that it stops without errors.
Copy all
InnoDB
data files (ibdata
files and.ibd
files) into a safe place.Copy all the
.frm
files forInnoDB
tables to a safe place.Copy all
InnoDB
log files (ib_logfile
files) to a safe place.Copy your
my.cnf
configuration file or files to a safe place.
Alternative Backup Types
In addition to making binary backups as just described, regularly
make dumps of your tables with mysqldump. A
binary file might be corrupted without you noticing it. Dumped
tables are stored into text files that are human-readable, so
spotting table corruption becomes easier. Also, because the format
is simpler, the chance for serious data corruption is smaller.
mysqldump also has a
--single-transaction
option for
making a consistent snapshot without locking out other clients.
See Section 6.3.1, “Establishing a Backup Policy”.
Replication works with InnoDB
tables,
so you can use MySQL replication capabilities to keep a copy of
your database at database sites requiring high availability.
Performing Recovery
To recover your InnoDB
database to the present
from the time at which the binary backup was made, you must run
your MySQL server with binary logging turned on, even before
taking the backup. To achieve point-in-time recovery after
restoring a backup, you can apply changes from the binary log that
occurred after the backup was made. See
Section 6.5, “Point-in-Time (Incremental) Recovery Using the Binary Log”.
To recover from a crash of your MySQL server, the only requirement
is to restart it. InnoDB
automatically checks
the logs and performs a roll-forward of the database to the
present. InnoDB
automatically rolls back
uncommitted transactions that were present at the time of the
crash. During recovery, mysqld displays output
something like this:
InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 13674004 InnoDB: Doing recovery: scanned up to log sequence number 0 13739520 InnoDB: Doing recovery: scanned up to log sequence number 0 13805056 InnoDB: Doing recovery: scanned up to log sequence number 0 13870592 InnoDB: Doing recovery: scanned up to log sequence number 0 13936128 ... InnoDB: Doing recovery: scanned up to log sequence number 0 20555264 InnoDB: Doing recovery: scanned up to log sequence number 0 20620800 InnoDB: Doing recovery: scanned up to log sequence number 0 20664692 InnoDB: 1 uncommitted transaction(s) which must be rolled back InnoDB: Starting rollback of uncommitted transactions InnoDB: Rolling back trx no 16745 InnoDB: Rolling back of trx no 16745 completed InnoDB: Rollback of uncommitted transactions completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Apply batch completed InnoDB: Started mysqld: ready for connections
If your database becomes corrupted or disk failure occurs, you must perform the recovery using a backup. In the case of corruption, first find a backup that is not corrupted. After restoring the base backup, do a point-in-time recovery from the binary log files using mysqlbinlog and mysql to restore the changes that occurred after the backup was made.
In some cases of database corruption it is enough just to dump,
drop, and re-create one or a few corrupt tables. You can use the
CHECK TABLE
SQL statement to check
whether a table is corrupt, although CHECK
TABLE
naturally cannot detect every possible kind of
corruption. You can use the Tablespace Monitor to check the
integrity of the file space management inside the tablespace
files.
In some cases, apparent database page corruption is actually due to the operating system corrupting its own file cache, and the data on disk may be okay. It is best first to try restarting your computer. Doing so may eliminate errors that appeared to be database page corruption.
InnoDB
crash recovery consists of several
steps. The first step, redo log application, is performed during
the initialization, before accepting any connections. If all
changes were flushed from the buffer pool to the tablespaces
(ibdata*
and *.ibd
files) at the time of the shutdown or crash, the redo log
application can be skipped. If the redo log files are missing at
startup, InnoDB
skips the redo log
application.
The remaining steps after redo log application do not depend on the redo log (other than for logging the writes) and are performed in parallel with normal processing. These include:
Rolling back incomplete transactions: Any transactions that were active at the time of crash or fast shutdown.
Insert buffer merge: Applying changes from the insert buffer tree (from the shared tablespace) to leaf pages of secondary indexes as the index pages are read to the buffer pool.
Purge: Deleting delete-marked records that are no longer visible for any active transaction.
Of these, only rollback of incomplete transactions is special to crash recovery. The insert buffer merge and the purge are performed during normal processing.
If there is database page corruption, you may want to dump your
tables from the database with
SELECT ... INTO
OUTFILE
. Usually, most of the data obtained in this
way is intact. However, it is possible that the corruption might
cause SELECT * FROM
statements or
tbl_name
InnoDB
background operations to crash or
assert, or even cause InnoDB
roll-forward
recovery to crash. In such cases, you can use the
innodb_force_recovery
option to
force the InnoDB
storage engine to start up
while preventing background operations from running, so that you
can dump your tables. For example, you can add the following
line to the [mysqld]
section of your option
file before restarting the server:
[mysqld] innodb_force_recovery = 4
innodb_force_recovery
is 0 by
default (normal startup without forced recovery). The
permissible nonzero values for
innodb_force_recovery
follow. A
larger number includes all precautions of smaller numbers. If
you can dump your tables with an option value of at most 4, then
you are relatively safe that only some data on corrupt
individual pages is lost. A value of 6 is more drastic because
database pages are left in an obsolete state, which in turn may
introduce more corruption into B-trees and other database
structures.
1
(SRV_FORCE_IGNORE_CORRUPT
)Let the server run even if it detects a corrupt page. Try to make
SELECT * FROM
jump over corrupt index records and pages, which helps in dumping tables.tbl_name
2
(SRV_FORCE_NO_BACKGROUND
)Prevent the main thread from running. If a crash would occur during the purge operation, this recovery value prevents it.
3
(SRV_FORCE_NO_TRX_UNDO
)Do not run transaction rollbacks after recovery.
4
(SRV_FORCE_NO_IBUF_MERGE
)Prevent insert buffer merge operations. If they would cause a crash, do not do them. Do not calculate table statistics.
5
(SRV_FORCE_NO_UNDO_LOG_SCAN
)Do not look at undo logs when starting the database:
InnoDB
treats even incomplete transactions as committed.6
(SRV_FORCE_NO_LOG_REDO
)Do not do the log roll-forward in connection with recovery.
With this value, you might not be able to do queries other than a basic
SELECT * FROM t
, with noWHERE
,ORDER BY
, or other clauses. More complex queries could encounter corrupted data structures and fail.If corruption within the table data prevents you from dumping the entire table contents, a query with an
ORDER BY
clause might be able to dump the portion of the table after the corrupted part.primary_key
DESC
The database must not otherwise be used with any
nonzero value of
innodb_force_recovery
.
As a safety measure, InnoDB
prevents users
from performing INSERT
,
UPDATE
, or
DELETE
operations when
innodb_force_recovery
is
greater than 0.
You can SELECT
from tables to
dump them, or DROP
or
CREATE
tables even if forced recovery is
used. If you know that a given table is causing a crash on
rollback, you can drop it. You can also use this to stop a
runaway rollback caused by a failing mass import or
ALTER TABLE
. You can kill the
mysqld process and set
innodb_force_recovery
to
3
to bring the database up without the
rollback, then DROP
the table that is causing
the runaway rollback.
Making your log files very large may reduce disk I/O during checkpointing. It often makes sense to set the total size of the log files as large as the buffer pool or even larger. Although in the past large log files could make crash recovery take excessive time, starting with MySQL 5.5, performance enhancements to crash recovery make it possible to use large log files with fast startup after a crash. (Strictly speaking, this performance improvement is available for MySQL 5.1 with the InnoDB Plugin 1.0.7 and higher. It is with MySQL 5.5 and InnoDB 1.1 that this improvement is available in the default InnoDB storage engine.)
How Checkpoint Processing Works
InnoDB
implements a checkpoint mechanism
known as “fuzzy” checkpointing.
InnoDB
flushes modified database pages from
the buffer pool in small batches. There is no need to flush the
buffer pool in one single batch, which would in practice stop
processing of user SQL statements during the checkpointing
process.
During crash recovery, InnoDB
looks for a
checkpoint label written to the log files. It knows that all
modifications to the database before the label are present in
the disk image of the database. Then InnoDB
scans the log files forward from the checkpoint, applying the
logged modifications to the database.
InnoDB
writes to its log files on a rotating
basis. It also writes checkpoint information to the first log
file at each checkpoint. All committed modifications that make
the database pages in the buffer pool different from the images
on disk must be available in the log files in case
InnoDB
has to do a recovery. This means that
when InnoDB
starts to reuse a log file, it
has to make sure that the database page images on disk contain
the modifications logged in the log file that
InnoDB
is going to reuse. In other words,
InnoDB
must create a checkpoint and this
often involves flushing of modified database pages to disk.
On Windows, InnoDB
always stores database and
table names internally in lowercase. To move databases in a binary
format from Unix to Windows or from Windows to Unix, create all
databases and tables using lowercase names. A convenient way to
accomplish this is to add the following line to the
[mysqld]
section of your
my.cnf
or my.ini
file
before creating any databases or tables:
[mysqld] lower_case_table_names=1
Like MyISAM
data files,
InnoDB
data and log files are binary-compatible
on all platforms having the same floating-point number format. You
can move an InnoDB
database simply by copying
all the relevant files listed in Section 13.3.7, “Backing Up and Recovering an InnoDB
Database”.
If the floating-point formats differ but you have not used
FLOAT
or
DOUBLE
data types in your tables,
then the procedure is the same: simply copy the relevant files. If
you use mysqldump to dump your tables on one
machine and then import the dump files on the other machine, it
does not matter whether the formats differ or your tables contain
floating-point data.
One way to increase performance is to switch off autocommit mode when importing data, assuming that the tablespace has enough space for the big rollback segment that the import transactions generate. Do the commit only after importing a whole table or a segment of a table.
- 13.3.9.1.
InnoDB
Lock Modes - 13.3.9.2. Consistent Nonlocking Reads
- 13.3.9.3.
SELECT ... FOR UPDATE
andSELECT ... LOCK IN SHARE MODE
Locking Reads - 13.3.9.4.
InnoDB
Record, Gap, and Next-Key Locks - 13.3.9.5. Avoiding the Phantom Problem Using Next-Key Locking
- 13.3.9.6. Locks Set by Different SQL Statements in
InnoDB
- 13.3.9.7. Implicit Transaction Commit and Rollback
- 13.3.9.8. Deadlock Detection and Rollback
- 13.3.9.9. How to Cope with Deadlocks
To implement a large-scale, busy, or highly reliable database application, to port substantial code from a different database system, or to push MySQL performance to the limits of the laws of physics, you must understand the notions of transactions and locking as they relate to the InnoDB storage engine.
In the InnoDB
transaction model, the goal is to
combine the best properties of a multi-versioning database with
traditional two-phase locking. InnoDB
does
locking on the row level and runs queries as nonlocking consistent
reads by default, in the style of Oracle. The lock information in
InnoDB
is stored so space-efficiently that lock
escalation is not needed: Typically, several users are permitted
to lock every row in InnoDB
tables, or any
random subset of the rows, without causing
InnoDB
memory exhaustion.
In InnoDB
, all user activity occurs inside a
transaction. If autocommit mode is enabled, each SQL statement
forms a single transaction on its own. By default, MySQL starts
the session for each new connection with autocommit enabled, so
MySQL does a commit after each SQL statement if that statement did
not return an error. If a statement returns an error, the commit
or rollback behavior depends on the error. See
Section 13.3.13, “InnoDB
Error Handling”.
A session that has autocommit enabled can perform a
multiple-statement transaction by starting it with an explicit
START
TRANSACTION
or
BEGIN
statement
and ending it with a COMMIT
or
ROLLBACK
statement. See Section 12.3.1, “START TRANSACTION
,
COMMIT
, and
ROLLBACK
Синтаксис”.
If autocommit mode is disabled within a session with SET
autocommit = 0
, the session always has a transaction
open. A COMMIT
or
ROLLBACK
statement ends the current transaction and a new one starts.
A COMMIT
means that the changes
made in the current transaction are made permanent and become
visible to other sessions. A
ROLLBACK
statement, on the other hand, cancels all modifications made by
the current transaction. Both
COMMIT
and
ROLLBACK
release
all InnoDB
locks that were set during the
current transaction.
In terms of the SQL:1992 transaction isolation levels, the default
InnoDB
level is
REPEATABLE READ
.
InnoDB
offers all four transaction isolation
levels described by the SQL standard:
READ UNCOMMITTED
,
READ COMMITTED
,
REPEATABLE READ
, and
SERIALIZABLE
.
A user can change the isolation level for a single session or for
all subsequent connections with the SET
TRANSACTION
statement. To set the server's default
isolation level for all connections, use the
--transaction-isolation
option on
the command line or in an option file. For detailed information
about isolation levels and level-setting syntax, see
Section 12.3.6, “SET TRANSACTION
Синтаксис”.
In row-level locking, InnoDB
normally uses
next-key locking. That means that besides index records,
InnoDB
can also lock the “gap”
preceding an index record to block insertions by other sessions in
the gap immediately before the index record. A next-key lock
refers to a lock that locks an index record and the gap before it.
A gap lock refers to a lock that locks only the gap before some
index record.
For more information about row-level locking, and the
circumstances under which gap locking is disabled, see
Section 13.3.9.4, “InnoDB
Record, Gap, and Next-Key Locks”.
InnoDB
implements standard row-level locking
where there are two types of locks:
A shared (
S
) lock permits a transaction to read a row.An exclusive (
X
) lock permits a transaction to update or delete a row.
If transaction T1
holds a shared
(S
) lock on row r
,
then requests from some distinct transaction
T2
for a lock on row r
are
handled as follows:
A request by
T2
for anS
lock can be granted immediately. As a result, bothT1
andT2
hold anS
lock onr
.A request by
T2
for anX
lock cannot be granted immediately.
If a transaction T1
holds an exclusive
(X
) lock on row r
,
a request from some distinct transaction T2
for a lock of either type on r
cannot be
granted immediately. Instead, transaction T2
has to wait for transaction T1
to release its
lock on row r
.
Additionally, InnoDB
supports
multiple granularity locking which permits
coexistence of record locks and locks on entire tables. To make
locking at multiple granularity levels practical, additional
types of locks called intention locks are
used. Intention locks are table locks in
InnoDB
. The idea behind intention locks is
for a transaction to indicate which type of lock (shared or
exclusive) it will require later for a row in that table. There
are two types of intention locks used in
InnoDB
(assume that transaction
T
has requested a lock of the indicated type
on table t
):
Intention shared (
IS
): TransactionT
intends to setS
locks on individual rows in tablet
.Intention exclusive (
IX
): TransactionT
intends to setX
locks on those rows.
For example, SELECT ...
LOCK IN SHARE MODE
sets an
IS
lock and
SELECT ... FOR
UPDATE
sets an IX
lock.
The intention locking protocol is as follows:
Before a transaction can acquire an
S
lock on a row in tablet
, it must first acquire anIS
or stronger lock ont
.Before a transaction can acquire an
X
lock on a row, it must first acquire anIX
lock ont
.
These rules can be conveniently summarized by means of the following lock type compatibility matrix.
X | IX | S | IS | |
---|---|---|---|---|
X | Conflict | Conflict | Conflict | Conflict |
IX | Conflict | Compatible | Conflict | Compatible |
S | Conflict | Conflict | Compatible | Compatible |
IS | Conflict | Compatible | Compatible | Compatible |
A lock is granted to a requesting transaction if it is compatible with existing locks, but not if it conflicts with existing locks. A transaction waits until the conflicting existing lock is released. If a lock request conflicts with an existing lock and cannot be granted because it would cause deadlock, an error occurs.
Thus, intention locks do not block anything except full table
requests (for example, LOCK TABLES ...
WRITE
). The main purpose of
IX
and IS
locks is to show that someone is locking a row, or going to lock
a row in the table.
The following example illustrates how an error can occur when a lock request would cause a deadlock. The example involves two clients, A and B.
First, client A creates a table containing one row, and then
begins a transaction. Within the transaction, A obtains an
S
lock on the row by selecting it in
share mode:
mysql>CREATE TABLE t (i INT) ENGINE = InnoDB;
Query OK, 0 rows affected (1.07 sec) mysql>INSERT INTO t (i) VALUES(1);
Query OK, 1 row affected (0.09 sec) mysql>START TRANSACTION;
Query OK, 0 rows affected (0.00 sec) mysql>SELECT * FROM t WHERE i = 1 LOCK IN SHARE MODE;
+------+ | i | +------+ | 1 | +------+ 1 row in set (0.10 sec)
Next, client B begins a transaction and attempts to delete the row from the table:
mysql>START TRANSACTION;
Query OK, 0 rows affected (0.00 sec) mysql>DELETE FROM t WHERE i = 1;
The delete operation requires an X
lock. The lock cannot be granted because it is incompatible with
the S
lock that client A holds, so
the request goes on the queue of lock requests for the row and
client B blocks.
Finally, client A also attempts to delete the row from the table:
mysql> DELETE FROM t WHERE i = 1;
ERROR 1213 (40001): Deadlock found when trying to get lock;
try restarting transaction
Deadlock occurs here because client A needs an
X
lock to delete the row. However,
that lock request cannot be granted because client B already has
a request for an X
lock and is
waiting for client A to release its S
lock. Nor can the S
lock held by A be
upgraded to an X
lock because of the
prior request by B for an X
lock. As
a result, InnoDB
generates an error for
client A and releases its locks. At that point, the lock request
for client B can be granted and B deletes the row from the
table.
A consistent read means that InnoDB
uses
multi-versioning to present to a query a snapshot of the
database at a point in time. The query sees the changes made by
transactions that committed before that point of time, and no
changes made by later or uncommitted transactions. The exception
to this rule is that the query sees the changes made by earlier
statements within the same transaction. This exception causes
the following anomaly: If you update some rows in a table, a
SELECT
sees the latest version of
the updated rows, but it might also see older versions of any
rows. If other sessions simultaneously update the same table,
the anomaly means that you might see the table in a state that
never existed in the database.
If the transaction isolation level is
REPEATABLE READ
(the default
level), all consistent reads within the same transaction read
the snapshot established by the first such read in that
transaction. You can get a fresher snapshot for your queries by
committing the current transaction and after that issuing new
queries.
With READ COMMITTED
isolation
level, each consistent read within a transaction sets and reads
its own fresh snapshot.
Consistent read is the default mode in which
InnoDB
processes
SELECT
statements in
READ COMMITTED
and
REPEATABLE READ
isolation
levels. A consistent read does not set any locks on the tables
it accesses, and therefore other sessions are free to modify
those tables at the same time a consistent read is being
performed on the table.
Suppose that you are running in the default
REPEATABLE READ
isolation
level. When you issue a consistent read (that is, an ordinary
SELECT
statement),
InnoDB
gives your transaction a timepoint
according to which your query sees the database. If another
transaction deletes a row and commits after your timepoint was
assigned, you do not see the row as having been deleted. Inserts
and updates are treated similarly.
You can advance your timepoint by committing your transaction
and then doing another SELECT
or
START TRANSACTION WITH
CONSISTENT SNAPSHOT
.
This is called multi-versioned concurrency control.
In the following example, session A sees the row inserted by B only when B has committed the insert and A has committed as well, so that the timepoint is advanced past the commit of B.
Session A Session B SET autocommit=0; SET autocommit=0; time | SELECT * FROM t; | empty set | INSERT INTO t VALUES (1, 2); | v SELECT * FROM t; empty set COMMIT; SELECT * FROM t; empty set COMMIT; SELECT * FROM t; --------------------- | 1 | 2 | --------------------- 1 row in set
If you want to see the “freshest” state of the
database, use either the READ
COMMITTED
isolation level or a locking read:
SELECT * FROM t LOCK IN SHARE MODE;
With READ COMMITTED
isolation
level, each consistent read within a transaction sets and reads
its own fresh snapshot. With LOCK IN SHARE
MODE
, a locking read occurs instead: A
SELECT
blocks until the transaction
containing the freshest rows ends (see
Section 13.3.9.3, “SELECT ... FOR UPDATE
and SELECT ... LOCK IN
SHARE MODE
Locking Reads”).
Consistent read does not work over certain DDL statements:
Consistent read does not work over
DROP TABLE
, because MySQL cannot use a table that has been dropped andInnoDB
destroys the table.Consistent read does not work over
ALTER TABLE
, because that statement makes a temporary copy of the original table and deletes the original table when the temporary copy is built. When you reissue a consistent read within a transaction, rows in the new table are not visible because those rows did not exist when the transaction's snapshot was taken.
The type of read varies for selects in clauses like
INSERT INTO ...
SELECT
, UPDATE
... (SELECT)
, and
CREATE TABLE ...
SELECT
that do not specify FOR
UPDATE
or LOCK IN SHARE MODE
:
By default,
InnoDB
uses stronger locks and theSELECT
part acts likeREAD COMMITTED
, where each consistent read, even within the same transaction, sets and reads its own fresh snapshot.To use a consistent read in such cases, enable the
innodb_locks_unsafe_for_binlog
option and set the isolation level of the transaction toREAD UNCOMMITTED
,READ COMMITTED
, orREPEATABLE READ
(that is, anything other thanSERIALIZABLE
). In this case, no locks are set on rows read from the selected table.
If you query data and then insert or update related data within
the same transaction, the regular SELECT
statement does not give enough protection. Other transactions
can update or delete the same rows you just queried.
InnoDB
supports two types of locking reads
that offer extra safety:
SELECT ... LOCK IN SHARE MODE
sets a shared mode lock on any rows that are read. Other sessions can read the rows, but cannot modify them until your transaction commits. If any of these rows were changed by another transaction that has not yet committed, your query waits until that transaction ends and then uses the latest values.SELECT ... FOR UPDATE
locks the rows and any associated index entries, the same as if you issued anUPDATE
statement for those rows. Other transactions are blocked from updating those rows, from doingSELECT ... LOCK IN SHARE MODE
, or from reading the data in certain transaction isolation levels. Consistent reads ignore any locks set on the records that exist in the read view. (Old versions of a record cannot be locked; they are reconstructed by applying undo logs on an in-memory copy of the record.)
These clauses are primarily useful when dealing with tree-structured or graph-structured data, either in a single table or split across multiple tables.
All locks set by LOCK IN SHARE MODE
and
FOR UPDATE
queries are released when the
transaction is committed or rolled back.
Locking of rows for update using SELECT FOR
UPDATE
only applies when autocommit is disabled
(either by beginning transaction with
START
TRANSACTION
or by setting
autocommit
to 0. If
autocommit is enabled, the rows matching the specification are
not locked.
Usage Examples
Suppose that you want to insert a new row into a table
child
, and make sure that the child row has a
parent row in table parent
. Your application
code can ensure referential integrity throughout this sequence
of operations.
First, use a consistent read to query the table
PARENT
and verify that the parent row exists.
Can you safely insert the child row to table
CHILD
? No, because some other session could
delete the parent row in the moment between your
SELECT
and your INSERT
,
without you being aware of it.
To avoid this potential issue, perform the
SELECT
using LOCK IN
SHARE MODE
:
SELECT * FROM parent WHERE NAME = 'Jones' LOCK IN SHARE MODE;
After the LOCK IN SHARE MODE
query returns
the parent 'Jones'
, you can safely add the
child record to the CHILD
table and commit
the transaction. Any transaction that tries to read or write to
the applicable row in the PARENT
table waits
until you are finished, that is, the data in all tables is in a
consistent state.
For another example, consider an integer counter field in a
table CHILD_CODES
, used to assign a unique
identifier to each child added to table
CHILD
. Do not use either consistent read or a
shared mode read to read the present value of the counter,
because two users of the database could see the same value for
the counter, and a duplicate-key error occurs if two
transactions attempt to add rows with the same identifier to the
CHILD
table.
Here, LOCK IN SHARE MODE
is not a good
solution because if two users read the counter at the same time,
at least one of them ends up in deadlock when it attempts to
update the counter.
Here are two ways to implement reading and incrementing the counter without interference from another transaction:
First update the counter by incrementing it by 1, then read it and use the new value in the
CHILD
table. Any other transaction that tries to read the counter waits until your transaction commits. If another transaction is in the middle of this same sequence, your transaction waits until the other one commits.First perform a locking read of the counter using
FOR UPDATE
, and then increment the counter:SELECT counter_field FROM child_codes FOR UPDATE; UPDATE child_codes SET counter_field = counter_field + 1;
A SELECT ... FOR
UPDATE
reads the latest available data, setting
exclusive locks on each row it reads. Thus, it sets the same
locks a searched SQL UPDATE
would
set on the rows.
The preceding description is merely an example of how
SELECT ... FOR
UPDATE
works. In MySQL, the specific task of
generating a unique identifier actually can be accomplished
using only a single access to the table:
UPDATE child_codes SET counter_field = LAST_INSERT_ID(counter_field + 1); SELECT LAST_INSERT_ID();
The SELECT
statement merely
retrieves the identifier information (specific to the current
connection). It does not access any table.
InnoDB
has several types of record-level
locks:
Record lock: This is a lock on an index record.
Gap lock: This is a lock on a gap between index records, or a lock on the gap before the first or after the last index record.
Next-key lock: This is a combination of a record lock on the index record and a gap lock on the gap before the index record.
Record locks always lock index records, even if a table is
defined with no indexes. For such cases,
InnoDB
creates a hidden clustered index and
uses this index for record locking. See
Section 13.3.11.1, “Clustered and Secondary Indexes”.
By default, InnoDB
operates in
REPEATABLE READ
transaction
isolation level and with the
innodb_locks_unsafe_for_binlog
system variable disabled. In this case,
InnoDB
uses next-key locks for searches and
index scans, which prevents phantom rows (see
Section 13.3.9.5, “Avoiding the Phantom Problem Using Next-Key Locking”).
Next-key locking combines index-row locking with gap locking.
InnoDB
performs row-level locking in such a
way that when it searches or scans a table index, it sets shared
or exclusive locks on the index records it encounters. Thus, the
row-level locks are actually index-record locks. In addition, a
next-key lock on an index record also affects the
“gap” before that index record. That is, a next-key
lock is an index-record lock plus a gap lock on the gap
preceding the index record. If one session has a shared or
exclusive lock on record R
in an index,
another session cannot insert a new index record in the gap
immediately before R
in the index order.
Suppose that an index contains the values 10, 11, 13, and 20.
The possible next-key locks for this index cover the following
intervals, where (
or )
denote exclusion of the interval endpoint and
[
or ]
denote inclusion of
the endpoint:
(negative infinity, 10] (10, 11] (11, 13] (13, 20] (20, positive infinity)
For the last interval, the next-key lock locks the gap above the largest value in the index and the “supremum” pseudo-record having a value higher than any value actually in the index. The supremum is not a real index record, so, in effect, this next-key lock locks only the gap following the largest index value.
The preceding example shows that a gap might span a single index value, multiple index values, or even be empty.
Gap locking is not needed for statements that lock rows using a
unique index to search for a unique row. (This does not include
the case that the search condition includes only some columns of
a multiple-column unique index; in that case, gap locking does
occur.) For example, if the id
column has a
unique index, the following statement uses only an index-record
lock for the row having id
value 100 and it
does not matter whether other sessions insert rows in the
preceding gap:
SELECT * FROM child WHERE id = 100;
If id
is not indexed or has a nonunique
index, the statement does lock the preceding gap.
A type of gap lock called an insertion intention gap lock is set
by INSERT
operations prior to row
insertion. This lock signals the intent to insert in such a way
that multiple transactions inserting into the same index gap
need not wait for each other if they are not inserting at the
same position within the gap. Suppose that there are index
records with values of 4 and 7. Separate transactions that
attempt to insert values of 5 and 6 each lock the gap between 4
and 7 with insert intention locks prior to obtaining the
exclusive lock on the inserted row, but do not block each other
because the rows are nonconflicting.
Gap locking can be disabled explicitly. This occurs if you
change the transaction isolation level to
READ COMMITTED
or enable the
innodb_locks_unsafe_for_binlog
system variable. Under these circumstances, gap locking is
disabled for searches and index scans and is used only for
foreign-key constraint checking and duplicate-key checking.
There are also other effects of using the
READ COMMITTED
isolation
level or enabling
innodb_locks_unsafe_for_binlog
:
Record locks for nonmatching rows are released after MySQL has
evaluated the WHERE
condition. For
UPDATE
statements,
InnoDB
does a
“semi-consistent” read, such that it returns the
latest committed version to MySQL so that MySQL can determine
whether the row matches the WHERE
condition
of the UPDATE
.
The so-called phantom problem occurs
within a transaction when the same query produces different sets
of rows at different times. For example, if a
SELECT
is executed twice, but
returns a row the second time that was not returned the first
time, the row is a “phantom” row.
Suppose that there is an index on the id
column of the child
table and that you want
to read and lock all rows from the table having an identifier
value larger than 100, with the intention of updating some
column in the selected rows later:
SELECT * FROM child WHERE id > 100 FOR UPDATE;
The query scans the index starting from the first record where
id
is bigger than 100. Let the table contain
rows having id
values of 90 and 102. If the
locks set on the index records in the scanned range do not lock
out inserts made in the gaps (in this case, the gap between 90
and 102), another session can insert a new row into the table
with an id
of 101. If you were to execute the
same SELECT
within the same
transaction, you would see a new row with an
id
of 101 (a “phantom”) in the
result set returned by the query. If we regard a set of rows as
a data item, the new phantom child would violate the isolation
principle of transactions that a transaction should be able to
run so that the data it has read does not change during the
transaction.
To prevent phantoms, InnoDB
uses an algorithm
called next-key locking that combines
index-row locking with gap locking. InnoDB
performs row-level locking in such a way that when it searches
or scans a table index, it sets shared or exclusive locks on the
index records it encounters. Thus, the row-level locks are
actually index-record locks. In addition, a next-key lock on an
index record also affects the “gap” before that
index record. That is, a next-key lock is an index-record lock
plus a gap lock on the gap preceding the index record. If one
session has a shared or exclusive lock on record
R
in an index, another session cannot insert
a new index record in the gap immediately before
R
in the index order.
When InnoDB
scans an index, it can also lock
the gap after the last record in the index. Just that happens in
the preceding example: To prevent any insert into the table
where id
would be bigger than 100, the locks
set by InnoDB
include a lock on the gap
following id
value 102.
You can use next-key locking to implement a uniqueness check in your application: If you read your data in share mode and do not see a duplicate for a row you are going to insert, then you can safely insert your row and know that the next-key lock set on the successor of your row during the read prevents anyone meanwhile inserting a duplicate for your row. Thus, the next-key locking enables you to “lock” the nonexistence of something in your table.
Gap locking can be disabled as discussed in
Section 13.3.9.4, “InnoDB
Record, Gap, and Next-Key Locks”. This may cause
phantom problems because other sessions can insert new rows into
the gaps when gap locking is disabled.
A locking read, an UPDATE
, or a
DELETE
generally set record locks
on every index record that is scanned in the processing of the
SQL statement. It does not matter whether there are
WHERE
conditions in the statement that would
exclude the row. InnoDB
does not remember the
exact WHERE
condition, but only knows which
index ranges were scanned. The locks are normally next-key locks
that also block inserts into the “gap” immediately
before the record. However, gap locking can be disabled
explicitly, which causes next-key locking not to be used. For
more information, see
Section 13.3.9.4, “InnoDB
Record, Gap, and Next-Key Locks”. The transaction
isolation level also can affect which locks are set; see
Section 12.3.6, “SET TRANSACTION
Синтаксис”.
If a secondary index is used in a search and index record locks
to be set are exclusive, InnoDB
also
retrieves the corresponding clustered index records and sets
locks on them.
Differences between shared and exclusive locks are described in
Section 13.3.9.1, “InnoDB
Lock Modes”.
If you have no indexes suitable for your statement and MySQL must scan the entire table to process the statement, every row of the table becomes locked, which in turn blocks all inserts by other users to the table. It is important to create good indexes so that your queries do not unnecessarily scan many rows.
For SELECT ... FOR
UPDATE
or
SELECT ... LOCK IN SHARE
MODE
, locks are acquired for scanned rows, and
expected to be released for rows that do not qualify for
inclusion in the result set (for example, if they do not meet
the criteria given in the WHERE
clause).
However, in some cases, rows might not be unlocked immediately
because the relationship between a result row and its original
source is lost during query execution. For example, in a
UNION
, scanned (and locked) rows
from a table might be inserted into a temporary table before
evaluation whether they qualify for the result set. In this
circumstance, the relationship of the rows in the temporary
table to the rows in the original table is lost and the latter
rows are not unlocked until the end of query execution.
InnoDB
sets specific types of locks as
follows.
SELECT ... FROM
is a consistent read, reading a snapshot of the database and setting no locks unless the transaction isolation level is set toSERIALIZABLE
. ForSERIALIZABLE
level, the search sets shared next-key locks on the index records it encounters.SELECT ... FROM ... LOCK IN SHARE MODE
sets shared next-key locks on all index records the search encounters.For index records the search encounters,
SELECT ... FROM ... FOR UPDATE
blocks other sessions from doingSELECT ... FROM ... LOCK IN SHARE MODE
or from reading in certain transaction isolation levels. Consistent reads will ignore any locks set on the records that exist in the read view.UPDATE ... WHERE ...
sets an exclusive next-key lock on every record the search encounters.DELETE FROM ... WHERE ...
sets an exclusive next-key lock on every record the search encounters.INSERT
sets an exclusive lock on the inserted row. This lock is an index-record lock, not a next-key lock (that is, there is no gap lock) and does not prevent other sessions from inserting into the gap before the inserted row.Prior to inserting the row, a type of gap lock called an insertion intention gap lock is set. This lock signals the intent to insert in such a way that multiple transactions inserting into the same index gap need not wait for each other if they are not inserting at the same position within the gap. Suppose that there are index records with values of 4 and 7. Separate transactions that attempt to insert values of 5 and 6 each lock the gap between 4 and 7 with insert intention locks prior to obtaining the exclusive lock on the inserted row, but do not block each other because the rows are nonconflicting.
If a duplicate-key error occurs, a shared lock on the duplicate index record is set. This use of a shared lock can result in deadlock should there be multiple sessions trying to insert the same row if another session already has an exclusive lock. This can occur if another session deletes the row. Suppose that an
InnoDB
tablet1
has the following structure:CREATE TABLE t1 (i INT, PRIMARY KEY (i)) ENGINE = InnoDB;
Now suppose that three sessions perform the following operations in order:
Session 1:
START TRANSACTION; INSERT INTO t1 VALUES(1);
Session 2:
START TRANSACTION; INSERT INTO t1 VALUES(1);
Session 3:
START TRANSACTION; INSERT INTO t1 VALUES(1);
Session 1:
ROLLBACK;
The first operation by session 1 acquires an exclusive lock for the row. The operations by sessions 2 and 3 both result in a duplicate-key error and they both request a shared lock for the row. When session 1 rolls back, it releases its exclusive lock on the row and the queued shared lock requests for sessions 2 and 3 are granted. At this point, sessions 2 and 3 deadlock: Neither can acquire an exclusive lock for the row because of the shared lock held by the other.
A similar situation occurs if the table already contains a row with key value 1 and three sessions perform the following operations in order:
Session 1:
START TRANSACTION; DELETE FROM t1 WHERE i = 1;
Session 2:
START TRANSACTION; INSERT INTO t1 VALUES(1);
Session 3:
START TRANSACTION; INSERT INTO t1 VALUES(1);
Session 1:
COMMIT;
The first operation by session 1 acquires an exclusive lock for the row. The operations by sessions 2 and 3 both result in a duplicate-key error and they both request a shared lock for the row. When session 1 commits, it releases its exclusive lock on the row and the queued shared lock requests for sessions 2 and 3 are granted. At this point, sessions 2 and 3 deadlock: Neither can acquire an exclusive lock for the row because of the shared lock held by the other.
INSERT ... ON DUPLICATE KEY UPDATE
differs from a simpleINSERT
in that an exclusive next-key lock rather than a shared lock is placed on the row to be updated when a duplicate-key error occurs.REPLACE
is done like anINSERT
if there is no collision on a unique key. Otherwise, an exclusive next-key lock is placed on the row to be replaced.INSERT INTO T SELECT ... FROM S WHERE ...
sets an exclusive index record without a gap lock on each row inserted intoT
. If the transaction isolation level isREAD COMMITTED
orinnodb_locks_unsafe_for_binlog
is enabled, and the transaction isolation level is notSERIALIZABLE
,InnoDB
does the search onS
as a consistent read (no locks). Otherwise,InnoDB
sets shared next-key locks on rows fromS
.InnoDB
has to set locks in the latter case: In roll-forward recovery from a backup, every SQL statement must be executed in exactly the same way it was done originally.CREATE TABLE ... SELECT ...
performs theSELECT
with shared next-key locks or as a consistent read, as forINSERT ... SELECT
.When a
SELECT
is used in the constructsREPLACE INTO t SELECT ... FROM s WHERE ...
orUPDATE t ... WHERE col IN (SELECT ... FROM s ...)
,InnoDB
sets shared next-key locks on rows from tables
.While initializing a previously specified
AUTO_INCREMENT
column on a table,InnoDB
sets an exclusive lock on the end of the index associated with theAUTO_INCREMENT
column. In accessing the auto-increment counter,InnoDB
uses a specificAUTO-INC
table lock mode where the lock lasts only to the end of the current SQL statement, not to the end of the entire transaction. Other sessions cannot insert into the table while theAUTO-INC
table lock is held; see Section 13.3.9, “TheInnoDB
Transaction Model and Locking”.InnoDB
fetches the value of a previously initializedAUTO_INCREMENT
column without setting any locks.If a
FOREIGN KEY
constraint is defined on a table, any insert, update, or delete that requires the constraint condition to be checked sets shared record-level locks on the records that it looks at to check the constraint.InnoDB
also sets these locks in the case where the constraint fails.LOCK TABLES
sets table locks, but it is the higher MySQL layer above theInnoDB
layer that sets these locks.InnoDB
is aware of table locks ifinnodb_table_locks = 1
(the default) andautocommit = 0
, and the MySQL layer aboveInnoDB
knows about row-level locks.Otherwise,
InnoDB
's automatic deadlock detection cannot detect deadlocks where such table locks are involved. Also, because in this case the higher MySQL layer does not know about row-level locks, it is possible to get a table lock on a table where another session currently has row-level locks. However, this does not endanger transaction integrity, as discussed in Section 13.3.9.8, “Deadlock Detection and Rollback”. See also Section 13.3.15, “Limits onInnoDB
Tables”.
By default, MySQL starts the session for each new connection
with autocommit mode enabled, so MySQL does a commit after each
SQL statement if that statement did not return an error. If a
statement returns an error, the commit or rollback behavior
depends on the error. See
Section 13.3.13, “InnoDB
Error Handling”.
If a session that has autocommit disabled ends without explicitly committing the final transaction, MySQL rolls back that transaction.
Some statements implicitly end a transaction, as if you had done
a COMMIT
before executing the
statement. For details, see Section 12.3.3, “Statements That Cause an Implicit Commit”.
InnoDB
automatically detects transaction
deadlocks and rolls back a transaction or transactions to break
the deadlock. InnoDB
tries to pick small
transactions to roll back, where the size of a transaction is
determined by the number of rows inserted, updated, or deleted.
InnoDB
is aware of table locks if
innodb_table_locks = 1
(the default) and
autocommit = 0
, and the MySQL
layer above it knows about row-level locks. Otherwise,
InnoDB
cannot detect deadlocks where a table
lock set by a MySQL LOCK TABLES
statement or a lock set by a storage engine other than
InnoDB
is involved. Resolve these situations
by setting the value of the
innodb_lock_wait_timeout
system
variable.
When InnoDB
performs a complete rollback of a
transaction, all locks set by the transaction are released.
However, if just a single SQL statement is rolled back as a
result of an error, some of the locks set by the statement may
be preserved. This happens because InnoDB
stores row locks in a format such that it cannot know afterward
which lock was set by which statement.
If a SELECT
calls a stored
function in a transaction, and a statement within the function
fails, that statement rolls back. Furthermore, if
ROLLBACK
is
executed after that, the entire transaction rolls back.
Deadlocks are a classic problem in transactional databases, but they are not dangerous unless they are so frequent that you cannot run certain transactions at all. Normally, you must write your applications so that they are always prepared to re-issue a transaction if it gets rolled back because of a deadlock.
InnoDB
uses automatic row-level locking. You
can get deadlocks even in the case of transactions that just
insert or delete a single row. That is because these operations
are not really “atomic”; they automatically set
locks on the (possibly several) index records of the row
inserted or deleted.
You can cope with deadlocks and reduce the likelihood of their occurrence with the following techniques:
Use
SHOW ENGINE INNODB STATUS
to determine the cause of the latest deadlock. That can help you to tune your application to avoid deadlocks.Always be prepared to re-issue a transaction if it fails due to deadlock. Deadlocks are not dangerous. Just try again.
Commit your transactions often. Small transactions are less prone to collision.
If you are using locking reads (
SELECT ... FOR UPDATE
orSELECT ... LOCK IN SHARE MODE
), try using a lower isolation level such asREAD COMMITTED
.Access your tables and rows in a fixed order. Then transactions form well-defined queues and do not deadlock.
Add well-chosen indexes to your tables. Then your queries need to scan fewer index records and consequently set fewer locks. Use
EXPLAIN SELECT
to determine which indexes the MySQL server regards as the most appropriate for your queries.Use less locking. If you can afford to permit a
SELECT
to return data from an old snapshot, do not add the clauseFOR UPDATE
orLOCK IN SHARE MODE
to it. Using theREAD COMMITTED
isolation level is good here, because each consistent read within the same transaction reads from its own fresh snapshot.If nothing else helps, serialize your transactions with table-level locks. The correct way to use
LOCK TABLES
with transactional tables, such asInnoDB
tables, is to begin a transaction withSET autocommit = 0
(notSTART TRANSACTION
) followed byLOCK TABLES
, and to not callUNLOCK TABLES
until you commit the transaction explicitly. For example, if you need to write to tablet1
and read from tablet2
, you can do this:SET autocommit=0; LOCK TABLES t1 WRITE, t2 READ, ...;
... do something with tables t1 and t2 here ...
COMMIT; UNLOCK TABLES;Table-level locks make your transactions queue nicely and avoid deadlocks.
Another way to serialize transactions is to create an auxiliary “semaphore” table that contains just a single row. Have each transaction update that row before accessing other tables. In that way, all transactions happen in a serial fashion. Note that the
InnoDB
instant deadlock detection algorithm also works in this case, because the serializing lock is a row-level lock. With MySQL table-level locks, the timeout method must be used to resolve deadlocks.
InnoDB
is a multi-versioned storage engine: it
keeps information about old versions of changed rows, to support
transactional features such as concurrency and rollback. This
information is stored in the tablespace in a data structure called
a rollback segment
(after an analogous data structure in Oracle).
InnoDB
uses the information in the rollback
segment to perform the undo operations needed in a transaction
rollback. It also uses the information to build earlier versions
of a row for a consistent read.
Internal Details of Multi-Versioning
Internally, InnoDB
adds three fields to each
row stored in the database. A 6-byte DB_TRX_ID
field indicates the transaction identifier for the last
transaction that inserted or updated the row. Also, a deletion is
treated internally as an update where a special bit in the row is
set to mark it as deleted. Each row also contains a 7-byte
DB_ROLL_PTR
field called the roll pointer. The
roll pointer points to an undo log record written to the rollback
segment. If the row was updated, the undo log record contains the
information necessary to rebuild the content of the row before it
was updated. A 6-byte DB_ROW_ID
field contains
a row ID that increases monotonically as new rows are inserted. If
InnoDB
generates a clustered index
automatically, the index contains row ID values. Otherwise, the
DB_ROW_ID
column does not appear in any index.
Undo logs in the rollback segment are divided into insert and
update undo logs. Insert undo logs are needed only in transaction
rollback and can be discarded as soon as the transaction commits.
Update undo logs are used also in consistent reads, but they can
be discarded only after there is no transaction present for which
InnoDB
has assigned a snapshot that in a
consistent read could need the information in the update undo log
to build an earlier version of a database row.
Guidelines for Managing Rollback Segments
Commit your transactions regularly, including those transactions
that issue only consistent reads. Otherwise,
InnoDB
cannot discard data from the update undo
logs, and the rollback segment may grow too big, filling up your
tablespace.
The physical size of an undo log record in the rollback segment is typically smaller than the corresponding inserted or updated row. You can use this information to calculate the space needed for your rollback segment.
In the InnoDB
multi-versioning scheme, a row is
not physically removed from the database immediately when you
delete it with an SQL statement. InnoDB
only
physically removes the corresponding row and its index records
when it discards the update undo log record written for the
deletion. This removal operation is called a
purge, and it is quite fast,
usually taking the same order of time as the SQL statement that
did the deletion.
If you insert and delete rows in smallish batches at about the
same rate in the table, the purge thread can start to lag behind
and the table can grow bigger and bigger because of all the
“dead” rows, making everything disk-bound and very
slow In such a case, throttle new row operations, and allocate
more resources to the purge thread by tuning the
innodb_max_purge_lag
system
variable. See Section 13.3.4, “InnoDB
Startup Options and System Variables” for more
information.
Role of the .frm File
MySQL stores its data dictionary information for tables in
.frm
files in database directories. This is
true for all MySQL storage engines, but every
InnoDB
table also has its own entry in the
InnoDB
internal data dictionary inside the
tablespace. When MySQL drops a table or a database, it has to
delete one or more .frm
files as well as the
corresponding entries inside the InnoDB
data
dictionary. Consequently, you cannot move
InnoDB
tables between databases simply by
moving the .frm
files.
Every InnoDB
table has a special index called
the clustered index
where the data for the rows is stored. Typically, the clustered
index is synonymous with the
primary key. To get the
best performance from queries, inserts, and other database
operations, you must understand how InnoDB uses the clustered
index to optimize the most common lookup and DML operations for
each table.
If you define a
PRIMARY KEY
on your table,InnoDB
uses it as the clustered index. Define a primary key for each table that you create. If there is no logical unique and non-null column or set of columns, add a new auto-increment column, whose values are filled in automatically.If you do not define a
PRIMARY KEY
for your table, MySQL locates the firstUNIQUE
index where all the key columns areNOT NULL
andInnoDB
uses it as the clustered index.If the table has no
PRIMARY KEY
or suitableUNIQUE
index,InnoDB
internally generates a hidden clustered index on a synthetic column containing row ID values. The rows are ordered by the ID thatInnoDB
assigns to the rows in such a table. The row ID is a 6-byte field that increases monotonically as new rows are inserted. Thus, the rows ordered by the row ID are physically in insertion order.
How the Clustered Index Speeds Up Queries
Accessing a row through the clustered index is fast because the
row data is on the same page where the index search leads. If a
table is large, the clustered index architecture often saves a
disk I/O operation when compared to storage organizations that
store row data using a different page from the index record.
(For example, MyISAM
uses one file for data
rows and another for index records.)
How Secondary Indexes Relate to the Clustered Index
All indexes other than the clustered index are known as
secondary indexes.
In InnoDB
, each record in a secondary index
contains the primary key columns for the row, as well as the
columns specified for the secondary index.
InnoDB
uses this primary key value to search
for the row in the clustered index.
If the primary key is long, the secondary indexes use more space, so it is advantageous to have a short primary key.
All InnoDB
indexes are B-trees where the
index records are stored in the leaf pages of the tree. The
default size of an index page is 16KB. When new records are
inserted, InnoDB
tries to leave 1/16 of the
page free for future insertions and updates of the index
records.
If index records are inserted in a sequential order (ascending
or descending), the resulting index pages are about 15/16 full.
If records are inserted in a random order, the pages are from
1/2 to 15/16 full. If the fill factor of an index page drops
below 1/2, InnoDB
tries to contract the index
tree to free the page.
Changing the page size is not a supported operation and there
is no guarantee that InnoDB
will
function normally with a page size other than 16KB. Problems
compiling or running InnoDB may occur. In particular,
ROW_FORMAT=COMPRESSED
in the Barracuda file
format assumes that the page size is at most 16KB and uses
14-bit pointers.
A version of InnoDB
built for one
page size cannot use data files or log files from a version
built for a different page size.
Database applications often insert new rows in the ascending order of the primary key. In this case, due to the layout of the clustered index in the same order as the primary key, insertions into an InnoDB table do not require random reads from a disk.
On the other hand, secondary indexes are usually nonunique, and
insertions into secondary indexes happen in a relatively random
order. In the same way, deletes and updates can affect data
pages that are not adjacent in secondary indexes. This would
cause a lot of random disk I/O operations without a special
mechanism used in InnoDB
.
When an index record is inserted, marked for deletion, or
deleted from a nonunique secondary index,
InnoDB
checks whether the secondary index
page is in the buffer pool. If that is the case,
InnoDB
applies the change directly to the
index page. If the index page is not found in the buffer pool,
InnoDB
records the change in a special
structure known as the insert
buffer. The insert buffer is kept small so that it fits
entirely in the buffer pool, and changes can be applied very
quickly. This process is known as
change buffering.
(Formerly, it applied only to inserts and was called insert
buffering. The data structure is still called the insert
buffer.)
Disk I/O for Flushing the Insert Buffer
Periodically, the insert buffer is merged into the secondary index trees in the database. Often, it is possible to merge several changes into the same page of the index tree, saving disk I/O operations. It has been measured that the insert buffer can speed up insertions into a table up to 15 times.
The insert buffer merging may continue to happen
after the transaction has been committed.
In fact, it may continue to happen after a server shutdown and
restart (see Section 13.3.7.2, “Forcing InnoDB
Recovery”).
Insert buffer merging may take many hours when many secondary
indexes must be updated and many rows have been inserted. During
this time, disk I/O will be increased, which can cause
significant slowdown on disk-bound queries. Another significant
background I/O operation is the purge thread (see
Section 13.3.10, “InnoDB
Multi-Versioning”).
The feature known as the
adaptive hash
index lets InnoDB
perform more like an
in-memory database on systems with appropriate combinations of
workload and ample memory for the buffer pool, without
sacrificing any transactional features or reliability.
If a table fits almost entirely in main memory, a hash index can
speed up queries by enabling direct lookup of any element,
turning the index value into a sort of pointer.
InnoDB
has a mechanism that monitors index
searches. If InnoDB
notices that queries
could benefit from building a hash index, it does so
automatically.
The hash index is always built based on an existing B-tree index
on the table. InnoDB
can build a hash index
on a prefix of any length of the key defined for the B-tree,
depending on the pattern of searches that
InnoDB
observes for the B-tree index. A hash
index can be partial, covering only those pages of the index
that are often accessed.
The physical row structure for an InnoDB
table depends on the row format specified when the table was
created. InnoDB
uses the
COMPACT
format by default, but the
REDUNDANT
format is available to retain
compatibility with older versions of MySQL. To check the row
format of an InnoDB
table, use
SHOW TABLE STATUS
.
The compact row format decreases row storage space by about 20% at the cost of increasing CPU use for some operations. If your workload is a typical one that is limited by cache hit rates and disk speed, compact format is likely to be faster. If the workload is a rare case that is limited by CPU speed, compact format might be slower.
Rows in InnoDB
tables that use
REDUNDANT
row format have the following
characteristics:
Each index record contains a six-byte header. The header is used to link together consecutive records, and also in row-level locking.
Records in the clustered index contain fields for all user-defined columns. In addition, there is a six-byte transaction ID field and a seven-byte roll pointer field.
If no primary key was defined for a table, each clustered index record also contains a six-byte row ID field.
Each secondary index record also contains all the primary key fields defined for the clustered index key that are not in the secondary index.
A record contains a pointer to each field of the record. If the total length of the fields in a record is less than 128 bytes, the pointer is one byte; otherwise, two bytes. The array of these pointers is called the record directory. The area where these pointers point is called the data part of the record.
Internally,
InnoDB
stores fixed-length character columns such asCHAR(10)
in a fixed-length format.InnoDB
does not truncate trailing spaces fromVARCHAR
columns.An SQL
NULL
value reserves one or two bytes in the record directory. Besides that, an SQLNULL
value reserves zero bytes in the data part of the record if stored in a variable length column. In a fixed-length column, it reserves the fixed length of the column in the data part of the record. Reserving the fixed space forNULL
values enables an update of the column fromNULL
to a non-NULL
value to be done in place without causing fragmentation of the index page.
Rows in InnoDB
tables that use
COMPACT
row format have the following
characteristics:
Each index record contains a five-byte header that may be preceded by a variable-length header. The header is used to link together consecutive records, and also in row-level locking.
The variable-length part of the record header contains a bit vector for indicating
NULL
columns. If the number of columns in the index that can beNULL
isN
, the bit vector occupiesCEILING(
bytes. (For example, if there are anywhere from 9 to 15 columns that can beN
/8)NULL
, the bit vector uses two bytes.) Columns that areNULL
do not occupy space other than the bit in this vector. The variable-length part of the header also contains the lengths of variable-length columns. Each length takes one or two bytes, depending on the maximum length of the column. If all columns in the index areNOT NULL
and have a fixed length, the record header has no variable-length part.For each non-
NULL
variable-length field, the record header contains the length of the column in one or two bytes. Two bytes will only be needed if part of the column is stored externally in overflow pages or the maximum length exceeds 255 bytes and the actual length exceeds 127 bytes. For an externally stored column, the two-byte length indicates the length of the internally stored part plus the 20-byte pointer to the externally stored part. The internal part is 768 bytes, so the length is 768+20. The 20-byte pointer stores the true length of the column.The record header is followed by the data contents of the non-
NULL
columns.Records in the clustered index contain fields for all user-defined columns. In addition, there is a six-byte transaction ID field and a seven-byte roll pointer field.
If no primary key was defined for a table, each clustered index record also contains a six-byte row ID field.
Each secondary index record also contains all the primary key fields defined for the clustered index key that are not in the secondary index. If any of these primary key fields are variable length, the record header for each secondary index will have a variable-length part to record their lengths, even if the secondary index is defined on fixed-length columns.
Internally,
InnoDB
stores fixed-length, fixed-width character columns such asCHAR(10)
in a fixed-length format.InnoDB
does not truncate trailing spaces fromVARCHAR
columns.Internally,
InnoDB
attempts to store UTF-8CHAR(
columns inN
)N
bytes by trimming trailing spaces. (WithREDUNDANT
row format, such columns occupy 3 ×N
bytes.) Reserving the minimum spaceN
in many cases enables column updates to be done in place without causing fragmentation of the index page.
The laws of physics dictate that it takes time and effort to read and write data. The ACID design model requires a certain amount of I/O that might seem redundant, but helps to ensure data reliability. Within these constraints, InnoDB tries to optimize the database work and the organization of disk files to minimize the amount of disk I/O. Sometimes, I/O is postponed until the database is not busy, or until everything needs to be brought to a consistent state, such as during a database restart after a crash.
InnoDB
uses asynchronous disk I/O where
possible, by creating a number of threads to handle I/O
operations, while permitting other database operations to
proceed while the I/O is still in progress. On Linux and Windows
platforms, InnoDB uses the available OS and library functions to
perform “native” asynchronous I/O. On other
platforms, InnoDB still uses I/O threads, but the threads may
actually wait for I/O requests to complete; this technique is
known as “simulated” asynchronous I/O.
Read-Ahead
If InnoDB can determine there is a high probability that data might be needed soon, it performs read-ahead operations to bring that data into the buffer pool so that it is available in memory. Making a few large read requests for contiguous data can be more efficient than making several small, spread-out requests. There are two read-ahead heuristics in InnoDB:
In sequential read-ahead, if
InnoDB
notices that the access pattern to a segment in the tablespace is sequential, it posts in advance a batch of reads of database pages to the I/O system.In random read-ahead, if
InnoDB
notices that some area in a tablespace seems to be in the process of being fully read into the buffer pool, it posts the remaining reads to the I/O system.
Doublewrite Buffer
InnoDB
uses a novel file flush technique
involving a structure called the
doublewrite
buffer. It adds safety to recovery following an operating
system crash or a power outage, and improves performance on most
varieties of Unix by reducing the need for
fsync()
operations.
Before writing pages to a data file, InnoDB
first writes them to a contiguous tablespace area called the
doublewrite buffer. Only after the write and the flush to the
doublewrite buffer has completed does InnoDB
write the pages to their proper positions in the data file. If
the operating system crashes in the middle of a page write,
InnoDB
can later find a good copy of the page
from the doublewrite buffer during recovery.
The data files that you define in the configuration file form
the InnoDB
system tablespace.
The files are logically concatenated to form the tablespace.
There is no striping in use. Currently, you cannot define where
within the tablespace your tables are allocated. However, in a
newly created tablespace, InnoDB
allocates
space starting from the first data file.
To avoid the issues that come with storing all tables and
indexes inside the system tablespace, you can turn on the
innodb_file_per_table
configuration option,
which stores each newly created table in a separate tablespace
file (with extension .ibd
). For tables stored
this way, there is less fragmentation within the disk file, and
when the table is truncated, the space is returned to the
operating system rather than still being reserved by InnoDB
within the system tablespace.
Pages, Extents, Segments, and Tablespaces
Each tablespace consists of database pages with a default size
of 16KB. The pages are grouped into extents of size 1MB (64
consecutive pages). The “files” inside a tablespace
are called segments in
InnoDB
. (These segments are different from
the “rollback segment”, which actually contains
many tablespace segments.)
When a segment grows inside the tablespace,
InnoDB
allocates the first 32 pages to it
individually. After that, InnoDB
starts to
allocate whole extents to the segment. InnoDB
can add up to 4 extents at a time to a large segment to ensure
good sequentiality of data.
Two segments are allocated for each index in
InnoDB
. One is for nonleaf nodes of the
B-tree, the other is for the leaf nodes. Keeping the leaf nodes
contiguous on disk enables better sequential I/O operations,
because these leaf nodes contain the actual table data.
Some pages in the tablespace contain bitmaps of other pages, and
therefore a few extents in an InnoDB
tablespace cannot be allocated to segments as a whole, but only
as individual pages.
When you ask for available free space in the tablespace by
issuing a SHOW TABLE STATUS
statement, InnoDB
reports the extents that
are definitely free in the tablespace. InnoDB
always reserves some extents for cleanup and other internal
purposes; these reserved extents are not included in the free
space.
When you delete data from a table, InnoDB
contracts the corresponding B-tree indexes. Whether the freed
space becomes available for other users depends on whether the
pattern of deletes frees individual pages or extents to the
tablespace. Dropping a table or deleting all rows from it is
guaranteed to release the space to other users, but remember
that deleted rows are physically removed only in an (automatic)
purge operation after they are no longer needed for transaction
rollbacks or consistent reads. (See
Section 13.3.10, “InnoDB
Multi-Versioning”.)
To see information about the tablespace, use the Tablespace
Monitor. See Section 13.3.14.2, “SHOW ENGINE INNODB
STATUS
and the InnoDB
Monitors”.
How Pages Relate to Table Rows
The maximum row length, except for variable-length columns
(VARBINARY
,
VARCHAR
,
BLOB
and
TEXT
), is slightly less than half
of a database page. That is, the maximum row length is about
8000 bytes. LONGBLOB
and
LONGTEXT
columns
must be less than 4GB, and the total row length, including
BLOB
and
TEXT
columns, must be less than
4GB.
If a row is less than half a page long, all of it is stored
locally within the page. If it exceeds half a page,
variable-length columns are chosen for external off-page storage
until the row fits within half a page. For a column chosen for
off-page storage, InnoDB
stores the first 768
bytes locally in the row, and the rest externally into overflow
pages. Each such column has its own list of overflow pages. The
768-byte prefix is accompanied by a 20-byte value that stores
the true length of the column and points into the overflow list
where the rest of the value is stored.
Random insertions into or deletions from a secondary index may cause the index to become fragmented. Fragmentation means that the physical ordering of the index pages on the disk is not close to the index ordering of the records on the pages, or that there are many unused pages in the 64-page blocks that were allocated to the index.
One symptom of fragmentation is that a table takes more space
than it “should” take. How much that is exactly, is
difficult to determine. All InnoDB
data and
indexes are stored in B-trees, and their fill factor may vary
from 50% to 100%. Another symptom of fragmentation is that a
table scan such as this takes more time than it
“should” take:
SELECT COUNT(*) FROM t WHERE a_non_indexed_column <> 12345;
The preceding query requires MySQL to scan the clustered index rather than a secondary index. Most disks can read 10MB/s to 50MB/s, which can be used to estimate how fast a table scan should be.
To speed up index scans, you can periodically perform a
“null” ALTER TABLE
operation, which causes MySQL to rebuild the table:
ALTER TABLE tbl_name
ENGINE=INNODB
Another way to perform a defragmentation operation is to use mysqldump to dump the table to a text file, drop the table, and reload it from the dump file.
If the insertions into an index are always ascending and records
are deleted only from the end, the InnoDB
filespace management algorithm guarantees that fragmentation in
the index does not occur.
Error handling in InnoDB
is not always the same
as specified in the SQL standard. According to the standard, any
error during an SQL statement should cause rollback of that
statement. InnoDB
sometimes rolls back only
part of the statement, or the whole transaction. The following
items describe how InnoDB
performs error
handling:
If you run out of file space in the tablespace, a MySQL
Table is full
error occurs andInnoDB
rolls back the SQL statement.A transaction deadlock causes
InnoDB
to roll back the entire transaction. Retry the whole transaction when this happens.A lock wait timeout causes
InnoDB
to roll back only the single statement that was waiting for the lock and encountered the timeout. (To have the entire transaction roll back, start the server with the--innodb_rollback_on_timeout
option.) Retry the statement if using the current behavior, or the entire transaction if using--innodb_rollback_on_timeout
.Both deadlocks and lock wait timeouts are normal on busy servers and it is necessary for applications to be aware that they may happen and handle them by retrying. You can make them less likely by doing as little work as possible between the first change to data during a transaction and the commit, so the locks are held for the shortest possible time and for the smallest possible number of rows. Sometimes splitting work between different transactions may be practical and helpful.
When a transaction rollback occurs due to a deadlock or lock wait timeout, it cancels the effect of the statements within the transaction. But if the start-transaction statement was
START TRANSACTION
orBEGIN
statement, rollback does not cancel that statement. Further SQL statements become part of the transaction until the occurrence ofCOMMIT
,ROLLBACK
, or some SQL statement that causes an implicit commit.A duplicate-key error rolls back the SQL statement, if you have not specified the
IGNORE
option in your statement.A
row too long error
rolls back the SQL statement.Other errors are mostly detected by the MySQL layer of code (above the
InnoDB
storage engine level), and they roll back the corresponding SQL statement. Locks are not released in a rollback of a single SQL statement.
During implicit rollbacks, as well as during the execution of an
explicit
ROLLBACK
SQL
statement, SHOW PROCESSLIST
displays Rolling back
in the
State
column for the relevant connection.
The following is a nonexhaustive list of common
InnoDB
-specific errors that you may
encounter, with information about why each occurs and how to
resolve the problem.
1005 (ER_CANT_CREATE_TABLE)
Cannot create table. If the error message refers to error 150, table creation failed because a foreign key constraint was not correctly formed. If the error message refers to error –1, table creation probably failed because the table includes a column name that matched the name of an internal
InnoDB
table.1016 (ER_CANT_OPEN_FILE)
Cannot find the
InnoDB
table from theInnoDB
data files, although the.frm
file for the table exists. See Section 13.3.14.4, “TroubleshootingInnoDB
Data Dictionary Operations”.1114 (ER_RECORD_FILE_FULL)
InnoDB
has run out of free space in the tablespace. Reconfigure the tablespace to add a new data file.1205 (ER_LOCK_WAIT_TIMEOUT)
Lock wait timeout expired. Transaction was rolled back.
1206 (ER_LOCK_TABLE_FULL)
The total number of locks exceeds the lock table size. To avoid this error, increase the value of
innodb_buffer_pool_size
. Within an individual application, a workaround may be to break a large operation into smaller pieces. For example, if the error occurs for a largeINSERT
, perform several smallerINSERT
operations.1213 (ER_LOCK_DEADLOCK)
Transaction deadlock. Rerun the transaction.
1216 (ER_NO_REFERENCED_ROW)
You are trying to add a row but there is no parent row, and a foreign key constraint fails. Add the parent row first.
1217 (ER_ROW_IS_REFERENCED)
You are trying to delete a parent row that has children, and a foreign key constraint fails. Delete the children first.
To print the meaning of an operating system error number, use the perror program that comes with the MySQL distribution.
The following table provides a list of some common Linux system error codes. For a more complete list, see Linux source code.
Number Macro Описание 1 EPERM
Operation not permitted 2 ENOENT
No such file or directory 3 ESRCH
No such process 4 EINTR
Interrupted system call 5 EIO
I/O error 6 ENXIO
No such device or address 7 E2BIG
Arg list too long 8 ENOEXEC
Exec format error 9 EBADF
Bad file number 10 ECHILD
No child processes 11 EAGAIN
Try again 12 ENOMEM
Out of memory 13 EACCES
Permission denied 14 EFAULT
Bad address 15 ENOTBLK
Block device required 16 EBUSY
Device or resource busy 17 EEXIST
File exists 18 EXDEV
Cross-device link 19 ENODEV
No such device 20 ENOTDIR
Not a directory 21 EISDIR
Is a directory 22 EINVAL
Invalid argument 23 ENFILE
File table overflow 24 EMFILE
Too many open files 25 ENOTTY
Inappropriate ioctl for device 26 ETXTBSY
Text file busy 27 EFBIG
File too large 28 ENOSPC
No space left on device 29 ESPIPE
Illegal seek 30 EROFS
Read-only file system 31 EMLINK
Too many links The following table provides a list of some common Windows system error codes. For a complete list, see the Microsoft Web site.
Number Macro Описание 1 ERROR_INVALID_FUNCTION
Incorrect function. 2 ERROR_FILE_NOT_FOUND
The system cannot find the file specified. 3 ERROR_PATH_NOT_FOUND
The system cannot find the path specified. 4 ERROR_TOO_MANY_OPEN_FILES
The system cannot open the file. 5 ERROR_ACCESS_DENIED
Access is denied. 6 ERROR_INVALID_HANDLE
The handle is invalid. 7 ERROR_ARENA_TRASHED
The storage control blocks were destroyed. 8 ERROR_NOT_ENOUGH_MEMORY
Not enough storage is available to process this command. 9 ERROR_INVALID_BLOCK
The storage control block address is invalid. 10 ERROR_BAD_ENVIRONMENT
The environment is incorrect. 11 ERROR_BAD_FORMAT
An attempt was made to load a program with an incorrect format. 12 ERROR_INVALID_ACCESS
The access code is invalid. 13 ERROR_INVALID_DATA
The data is invalid. 14 ERROR_OUTOFMEMORY
Not enough storage is available to complete this operation. 15 ERROR_INVALID_DRIVE
The system cannot find the drive specified. 16 ERROR_CURRENT_DIRECTORY
The directory cannot be removed. 17 ERROR_NOT_SAME_DEVICE
The system cannot move the file to a different disk drive. 18 ERROR_NO_MORE_FILES
There are no more files. 19 ERROR_WRITE_PROTECT
The media is write protected. 20 ERROR_BAD_UNIT
The system cannot find the device specified. 21 ERROR_NOT_READY
The device is not ready. 22 ERROR_BAD_COMMAND
The device does not recognize the command. 23 ERROR_CRC
Data error (cyclic redundancy check). 24 ERROR_BAD_LENGTH
The program issued a command but the command length is incorrect. 25 ERROR_SEEK
The drive cannot locate a specific area or track on the disk. 26 ERROR_NOT_DOS_DISK
The specified disk or diskette cannot be accessed. 27 ERROR_SECTOR_NOT_FOUND
The drive cannot find the sector requested. 28 ERROR_OUT_OF_PAPER
The printer is out of paper. 29 ERROR_WRITE_FAULT
The system cannot write to the specified device. 30 ERROR_READ_FAULT
The system cannot read from the specified device. 31 ERROR_GEN_FAILURE
A device attached to the system is not functioning. 32 ERROR_SHARING_VIOLATION
The process cannot access the file because it is being used by another process. 33 ERROR_LOCK_VIOLATION
The process cannot access the file because another process has locked a portion of the file. 34 ERROR_WRONG_DISK
The wrong diskette is in the drive. Insert %2 (Volume Serial Number: %3) into drive %1. 36 ERROR_SHARING_BUFFER_EXCEEDED
Too many files opened for sharing. 38 ERROR_HANDLE_EOF
Reached the end of the file. 39 ERROR_HANDLE_DISK_FULL
The disk is full. 87 ERROR_INVALID_PARAMETER
The parameter is incorrect. 112 ERROR_DISK_FULL
The disk is full. 123 ERROR_INVALID_NAME
The file name, directory name, or volume label syntax is incorrect. 1450 ERROR_NO_SYSTEM_RESOURCES
Insufficient system resources exist to complete the requested service.
With InnoDB
becoming the default storage
engine in MySQL 5.5 and higher, the tips and guidelines for
InnoDB
tables are now part of the main
optimization chapter. See Section 7.5, “Optimizing for InnoDB
Tables”.
InnoDB
Monitors provide information about the
InnoDB
internal state. This information is
useful for performance tuning. Each Monitor can be enabled by
creating a table with a special name, which causes
InnoDB
to write Monitor output periodically.
Also, output for the standard InnoDB
Monitor
is available on demand through the
SHOW ENGINE INNODB
STATUS
SQL statement.
There are several types of InnoDB
Monitors:
The standard
InnoDB
Monitor displays the following types of information:Table and record locks held by each active transaction
Lock waits of a transactions
Semaphore waits of threads
Pending file I/O requests
Buffer pool statistics
Purge and insert buffer merge activity of the main
InnoDB
thread
For a discussion of
InnoDB
lock modes, see Section 13.3.9.1, “InnoDB
Lock Modes”.To enable the standard
InnoDB
Monitor for periodic output, create a table namedinnodb_monitor
. To obtain Monitor output on demand, use theSHOW ENGINE INNODB STATUS
SQL statement to fetch the output to your client program. If you are using the mysql interactive client, the output is more readable if you replace the usual semicolon statement terminator with\G
:mysql>
SHOW ENGINE INNODB STATUS\G
The
InnoDB
Lock Monitor is like the standard Monitor but also provides extensive lock information. To enable this Monitor for periodic output, create a table namedinnodb_lock_monitor
.The
InnoDB
Tablespace Monitor prints a list of file segments in the shared tablespace and validates the tablespace allocation data structures. To enable this Monitor for periodic output, create a table namedinnodb_tablespace_monitor
.The
InnoDB
Table Monitor prints the contents of theInnoDB
internal data dictionary. To enable this Monitor for periodic output, create a table namedinnodb_table_monitor
.
To enable an InnoDB
Monitor for periodic
output, use a CREATE TABLE
statement to
create the table associated with the Monitor. For example, to
enable the standard InnoDB
Monitor, create
the innodb_monitor
table:
CREATE TABLE innodb_monitor (a INT) ENGINE=INNODB;
To stop the Monitor, drop the table:
DROP TABLE innodb_monitor;
The CREATE TABLE
syntax is just a
way to pass a command to the InnoDB
engine
through MySQL's SQL parser: The only things that matter are the
table name innodb_monitor
and that it be an
InnoDB
table. The structure of the table is
not relevant at all for the InnoDB
Monitor.
If you shut down the server, the Monitor does not restart
automatically when you restart the server. Drop the Monitor
table and issue a new CREATE
TABLE
statement to start the Monitor. (This syntax may
change in a future release.)
The PROCESS
privilege is required
to start or stop the InnoDB
Monitor tables.
When you enable InnoDB
Monitors for periodic
output, InnoDB
writes their output to the
mysqld server standard error output
(stderr
). In this case, no output is sent to
clients. When switched on, InnoDB
Monitors
print data about every 15 seconds. Server output usually is
directed to the error log (see Section 5.2.2, “The Error Log”).
This data is useful in performance tuning. On Windows, start the
server from a command prompt in a console window with the
--console
option if you want to
direct the output to the window rather than to the error log.
InnoDB
sends diagnostic output to
stderr
or to files rather than to
stdout
or fixed-size memory buffers, to avoid
potential buffer overflows. As a side effect, the output of
SHOW ENGINE INNODB
STATUS
is written to a status file in the MySQL data
directory every fifteen seconds. The name of the file is
innodb_status.
,
where pid
pid
is the server process ID.
InnoDB
removes the file for a normal
shutdown. If abnormal shutdowns have occurred, instances of
these status files may be present and must be removed manually.
Before removing them, you might want to examine them to see
whether they contain useful information about the cause of
abnormal shutdowns. The
innodb_status.
file is created only if the configuration option
pid
innodb-status-file=1
is set.
InnoDB
Monitors should be enabled only when
you actually want to see Monitor information because output
generation does result in some performance decrement. Also, if
you enable monitor output by creating the associated table, your
error log may become quite large if you forget to remove the
table later.
For additional information about InnoDB
monitors, see:
Mark Leith: InnoDB Table and Tablespace Monitors
Each monitor begins with a header containing a timestamp and the monitor name. For example:
================================================ 090407 12:06:19 INNODB TABLESPACE MONITOR OUTPUT ================================================
The header for the standard Monitor (INNODB MONITOR
OUTPUT
) is also used for the Lock Monitor because the
latter produces the same output with the addition of extra lock
information.
The following sections describe the output for each Monitor.
The Lock Monitor is the same as the standard Monitor except
that it includes additional lock information. Enabling either
monitor for periodic output by creating the associated
InnoDB
table turns on the same output
stream, but the stream includes the extra information if the
Lock Monitor is enabled. For example, if you create the
innodb_monitor
and
innodb_lock_monitor
tables, that turns on a
single output stream. The stream includes extra lock
information until you disable the Lock Monitor by removing the
innodb_lock_monitor
table.
Пример InnoDB
Monitor output:
mysql> SHOW ENGINE INNODB STATUS\G
*************************** 1. row ***************************
Status:
=====================================
030709 13:00:59 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 18 seconds
----------
BACKGROUND THREAD
----------
srv_master_thread loops: 53 1_second, 44 sleeps, 5 10_second, 7 background,
7 flush
srv_master_thread log flush and writes: 48
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 413452, signal count 378357
--Thread 32782 has waited at btr0sea.c line 1477 for 0.00 seconds the
semaphore: X-lock on RW-latch at 41a28668 created in file btr0sea.c line 135
a writer (thread id 32782) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file btr0sea.c line 731
Last time write locked in file btr0sea.c line 1347
Mutex spin waits 0, rounds 0, OS waits 0
RW-shared spins 2, rounds 60, OS waits 2
RW-excl spins 0, rounds 0, OS waits 0
Spin rounds per wait: 0.00 mutex, 20.00 RW-shared, 0.00 RW-excl
------------------------
LATEST FOREIGN KEY ERROR
------------------------
030709 13:00:59 Transaction:
TRANSACTION 0 290328284, ACTIVE 0 sec, process no 3195
inserting
15 lock struct(s), heap size 2496, undo log entries 9
MySQL thread id 25, query id 4668733 localhost heikki update
insert into ibtest11a (D, B, C) values (5, 'khDk' ,'khDk')
Foreign key constraint fails for table test/ibtest11a:
,
CONSTRAINT `0_219242` FOREIGN KEY (`A`, `D`) REFERENCES `ibtest11b` (`A`,
`D`) ON DELETE CASCADE ON UPDATE CASCADE
Trying to add in child table, in index PRIMARY tuple:
0: len 4; hex 80000101; asc ....;; 1: len 4; hex 80000005; asc ....;; 2:
len 4; hex 6b68446b; asc khDk;; 3: len 6; hex 0000114e0edc; asc ...N..;; 4:
len 7; hex 00000000c3e0a7; asc .......;; 5: len 4; hex 6b68446b; asc khDk;;
But in parent table test/ibtest11b, in index PRIMARY,
the closest match we can find is record:
RECORD: info bits 0 0: len 4; hex 8000015b; asc ...[;; 1: len 4; hex
80000005; asc ....;; 2: len 3; hex 6b6864; asc khd;; 3: len 6; hex
0000111ef3eb; asc ......;; 4: len 7; hex 800001001e0084; asc .......;; 5:
len 3; hex 6b6864; asc khd;;
------------------------
LATEST DETECTED DEADLOCK
------------------------
030709 12:59:58
*** (1) TRANSACTION:
TRANSACTION 0 290252780, ACTIVE 1 sec, process no 3185
inserting
LOCK WAIT 3 lock struct(s), heap size 320, undo log entries 146
MySQL thread id 21, query id 4553379 localhost heikki update
INSERT INTO alex1 VALUES(86, 86, 794,'aA35818','bb','c79166','d4766t',
'e187358f','g84586','h794',date_format('2001-04-03 12:54:22','%Y-%m-%d
%H:%i'),7
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
symbole trx id 0 290252780 lock mode S waiting
Record lock, heap no 324 RECORD: info bits 0 0: len 7; hex 61613335383138;
asc aa35818;; 1:
*** (2) TRANSACTION:
TRANSACTION 0 290251546, ACTIVE 2 sec, process no 3190
inserting
130 lock struct(s), heap size 11584, undo log entries 437
MySQL thread id 23, query id 4554396 localhost heikki update
REPLACE INTO alex1 VALUES(NULL, 32, NULL,'aa3572','','c3572','d6012t','',
NULL,'h396', NULL, NULL, 7.31,7.31,7.31,200)
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
symbole trx id 0 290251546 lock_mode X locks rec but not gap
Record lock, heap no 324 RECORD: info bits 0 0: len 7; hex 61613335383138;
asc aa35818;; 1:
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
symbole trx id 0 290251546 lock_mode X locks gap before rec insert intention
waiting
Record lock, heap no 82 RECORD: info bits 0 0: len 7; hex 61613335373230;
asc aa35720;; 1:
*** WE ROLL BACK TRANSACTION (1)
------------
TRANSACTIONS
------------
Trx id counter 0 290328385
Purge done for trx's n:o < 0 290315608 undo n:o < 0 17
History list length 20
Total number of lock structs in row lock hash table 70
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0 0, not started, process no 3491
MySQL thread id 32, query id 4668737 localhost heikki
show innodb status
---TRANSACTION 0 290328384, ACTIVE 0 sec, process no 3205
38929 inserting
1 lock struct(s), heap size 320
MySQL thread id 29, query id 4668736 localhost heikki update
insert into speedc values (1519229,1, 'hgjhjgghggjgjgjgjgjggjgjgjgjgjgggjgjg
jlhhgghggggghhjhghgggggghjhghghghghghhhhghghghjhhjghjghjkghjghjghjghjfhjfh
---TRANSACTION 0 290328383, ACTIVE 0 sec, process no 3180
28684 committing
1 lock struct(s), heap size 320, undo log entries 1
MySQL thread id 19, query id 4668734 localhost heikki update
insert into speedcm values (1603393,1, 'hgjhjgghggjgjgjgjgjggjgjgjgjgjgggjgj
gjlhhgghggggghhjhghgggggghjhghghghghghhhhghghghjhhjghjghjkghjghjghjghjfhjf
---TRANSACTION 0 290328327, ACTIVE 0 sec, process no 3200
36880 starting index read
LOCK WAIT 2 lock struct(s), heap size 320
MySQL thread id 27, query id 4668644 localhost heikki Searching rows for
update
update ibtest11a set B = 'kHdkkkk' where A = 89572
------- TRX HAS BEEN WAITING 0 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 65556 n bits 232 table test/ibtest11a index
PRIMARY trx id 0 290328327 lock_mode X waiting
Record lock, heap no 1 RECORD: info bits 0 0: len 9; hex 73757072656d756d00;
asc supremum.;;
------------------
---TRANSACTION 0 290328284, ACTIVE 0 sec, process no 3195
34831 rollback of SQL statement
ROLLING BACK 14 lock struct(s), heap size 2496, undo log entries 9
MySQL thread id 25, query id 4668733 localhost heikki update
insert into ibtest11a (D, B, C) values (5, 'khDk' ,'khDk')
---TRANSACTION 0 290327208, ACTIVE 1 sec, process no 3190
32782
58 lock struct(s), heap size 5504, undo log entries 159
MySQL thread id 23, query id 4668732 localhost heikki update
REPLACE INTO alex1 VALUES(86, 46, 538,'aa95666','bb','c95666','d9486t',
'e200498f','g86814','h538',date_format('2001-04-03 12:54:22','%Y-%m-%d
%H:%i'),
---TRANSACTION 0 290323325, ACTIVE 3 sec, process no 3185
30733 inserting
4 lock struct(s), heap size 1024, undo log entries 165
MySQL thread id 21, query id 4668735 localhost heikki update
INSERT INTO alex1 VALUES(NULL, 49, NULL,'aa42837','','c56319','d1719t','',
NULL,'h321', NULL, NULL, 7.31,7.31,7.31,200)
--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (write thread)
Pending normal aio reads: 0, aio writes: 0,
ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
151671 OS file reads, 94747 OS file writes, 8750 OS fsyncs
25.44 reads/s, 18494 avg bytes/read, 17.55 writes/s, 2.33 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf for space 0: size 1, free list len 19, seg size 21,
85004 inserts, 85004 merged recs, 26669 merges
Hash table size 207619, used cells 14461, node heap has 16 buffer(s)
1877.67 hash searches/s, 5121.10 non-hash searches/s
---
LOG
---
Log sequence number 18 1212842764
Log flushed up to 18 1212665295
Last checkpoint at 18 1135877290
0 pending log writes, 0 pending chkp writes
4341 log i/o's done, 1.22 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 84966343; in additional pool allocated 1402624
Buffer pool size 3200
Free buffers 110
Database pages 3074
Modified db pages 2674
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages read 171380, created 51968, written 194688
28.72 reads/s, 20.72 creates/s, 47.55 writes/s
Buffer pool hit rate 999 / 1000
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
Main thread process no. 3004, id 7176, state: purging
Number of rows inserted 3738558, updated 127415, deleted 33707, read 755779
1586.13 inserts/s, 50.89 updates/s, 28.44 deletes/s, 107.88 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
InnoDB
Monitor output is limited to 1MB
when produced using the
SHOW ENGINE
INNODB STATUS
statement. This limit does not apply
to output written to the server's error output.
Some notes on the output sections:
BACKGROUND
THREAD
The srv_master_thread
lines shows work done
by the main background thread.
SEMAPHORES
This section reports threads waiting for a semaphore and
statistics on how many times threads have needed a spin or a
wait on a mutex or a rw-lock semaphore. A large number of
threads waiting for semaphores may be a result of disk I/O, or
contention problems inside InnoDB
.
Contention can be due to heavy parallelism of queries or
problems in operating system thread scheduling. Setting the
innodb_thread_concurrency
system variable smaller than the default value might help in
such situations. The Spin rounds per wait
line shows the number of spinlock rounds per OS wait for a
mutex.
LATEST FOREIGN KEY
ERROR
This section provides information about the most recent foreign key constraint error. It is not present if no such error has occurred. The contents include the statement that failed as well as information about the constraint that failed and the referenced and referencing tables.
LATEST DETECTED
DEADLOCK
This section provides information about the most recent
deadlock. It is not present if no deadlock has occurred. The
contents show which transactions are involved, the statement
each was attempting to execute, the locks they have and need,
and which transaction InnoDB
decided to
roll back to break the deadlock. The lock modes reported in
this section are explained in
Section 13.3.9.1, “InnoDB
Lock Modes”.
TRANSACTIONS
If this section reports lock waits, your applications might have lock contention. The output can also help to trace the reasons for transaction deadlocks.
FILE I/O
This section provides information about threads that
InnoDB
uses to perform various types of
I/O. The first few of these are dedicated to general
InnoDB
processing. The contents also
display information for pending I/O operations and statistics
for I/O performance.
The number of these threads are controlled by the
innodb_read_io_threads
and
innodb_write_io_threads
parameters. See Section 13.3.4, “InnoDB
Startup Options and System Variables”.
INSERT BUFFER AND ADAPTIVE HASH
INDEX
This section shows the status of the InnoDB
insert buffer and adaptive hash index. (See
Section 13.3.11.3, “Insert Buffering”, and
Section 13.3.11.4, “Adaptive Hash Indexes”.) The contents include
the number of operations performed for each, plus statistics
for hash index performance.
LOG
This section displays information about the
InnoDB
log. The contents include the
current log sequence number, how far the log has been flushed
to disk, and the position at which InnoDB
last took a checkpoint. (See
Section 13.3.7.3, “InnoDB
Checkpoints”.) The section also
displays information about pending writes and write
performance statistics.
BUFFER POOL AND
MEMORY
This section gives you statistics on pages read and written. You can calculate from these numbers how many data file I/O operations your queries currently are doing.
For additional information about the operation of the buffer
pool, see Section 7.9.1, “The InnoDB
Buffer Pool”.
ROW
OPERATIONS
This section shows what the main thread is doing, including the number and performance rate for each type of row operation.
In MySQL 5.5, output from the standard Monitor includes additional sections compared to the output for previous versions. For details, see Section 1.4.3, “Diagnostic and Monitoring Capabilities”.
The InnoDB
Tablespace Monitor prints
information about the file segments in the shared tablespace
and validates the tablespace allocation data structures. If
you use individual tablespaces by enabling
innodb_file_per_table
, the
Tablespace Monitor does not describe those tablespaces.
Пример InnoDB
Tablespace Monitor output:
================================================ 090408 21:28:09 INNODB TABLESPACE MONITOR OUTPUT ================================================ FILE SPACE INFO: id 0 size 13440, free limit 3136, free extents 28 not full frag extents 2: used pages 78, full frag extents 3 first seg id not used 0 23845 SEGMENT id 0 1 space 0; page 2; res 96 used 46; full ext 0 fragm pages 32; free extents 0; not full extents 1: pages 14 SEGMENT id 0 2 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 3 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 ... SEGMENT id 0 15 space 0; page 2; res 160 used 160; full ext 2 fragm pages 32; free extents 0; not full extents 0: pages 0 SEGMENT id 0 488 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 17 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 ... SEGMENT id 0 171 space 0; page 2; res 592 used 481; full ext 7 fragm pages 16; free extents 0; not full extents 2: pages 17 SEGMENT id 0 172 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 173 space 0; page 2; res 96 used 44; full ext 0 fragm pages 32; free extents 0; not full extents 1: pages 12 ... SEGMENT id 0 601 space 0; page 2; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 NUMBER of file segments: 73 Validating tablespace Validation ok --------------------------------------- END OF INNODB TABLESPACE MONITOR OUTPUT =======================================
The Tablespace Monitor output includes information about the shared tablespace as a whole, followed by a list containing a breakdown for each segment within the tablespace.
The tablespace consists of database pages with a default size of 16KB. The pages are grouped into extents of size 1MB (64 consecutive pages).
The initial part of the output that displays overall tablespace information has this format:
FILE SPACE INFO: id 0 size 13440, free limit 3136, free extents 28 not full frag extents 2: used pages 78, full frag extents 3 first seg id not used 0 23845
Overall tablespace information includes these values:
id
: The tablespace ID. A value of 0 refers to the shared tablespace.size
: The current tablespace size in pages.free limit
: The minimum page number for which the free list has not been initialized. Pages at or above this limit are free.free extents
: The number of free extents.not full frag extents
,used pages
: The number of fragment extents that are not completely filled, and the number of pages in those extents that have been allocated.full frag extents
: The number of completely full fragment extents.first seg id not used
: The first unused segment ID.
Individual segment information has this format:
SEGMENT id 0 15 space 0; page 2; res 160 used 160; full ext 2 fragm pages 32; free extents 0; not full extents 0: pages 0
Segment information includes these values:
id
: The segment ID.
space
, page
: The
tablespace number and page within the tablespace where the
segment “inode” is located. A tablespace number
of 0 indicates the shared tablespace.
InnoDB
uses inodes to keep track of
segments in the tablespace. The other fields displayed for a
segment (id
, res
, and so
forth) are derived from information in the inode.
res
: The number of pages allocated
(reserved) for the segment.
used
: The number of allocated pages in use
by the segment.
full ext
: The number of extents allocated
for the segment that are completely used.
fragm pages
: The number of initial pages
that have been allocated to the segment.
free extents
: The number of extents
allocated for the segment that are completely unused.
not full extents
: The number of extents
allocated for the segment that are partially used.
pages
: The number of pages used within the
not-full extents.
When a segment grows, it starts as a single page, and
InnoDB
allocates the first pages for it
individually, up to 32 pages (this is the fragm
pages
value). After that, InnoDB
allocates complete 64-page extents. InnoDB
can add up to 4 extents at a time to a large segment to ensure
good sequentiality of data.
For the example segment shown earlier, it has 32 fragment pages, plus 2 full extents (64 pages each), for a total of 160 pages used out of 160 pages allocated. The following segment has 32 fragment pages and one partially full extent using 14 pages for a total of 46 pages used out of 96 pages allocated:
SEGMENT id 0 1 space 0; page 2; res 96 used 46; full ext 0 fragm pages 32; free extents 0; not full extents 1: pages 14
It is possible for a segment that has extents allocated to it
to have a fragm pages
value less than 32 if
some of the individual pages have been deallocated subsequent
to extent allocation.
The InnoDB
Table Monitor prints the
contents of the InnoDB
internal data
dictionary.
The output contains one section per table. The
SYS_FOREIGN
and
SYS_FOREIGN_COLS
sections are for internal
data dictionary tables that maintain information about foreign
keys. There are also sections for the Table Monitor table and
each user-created InnoDB
table. Suppose
that the following two tables have been created in the
test
database:
CREATE TABLE parent ( par_id INT NOT NULL, fname CHAR(20), lname CHAR(20), PRIMARY KEY (par_id), UNIQUE INDEX (lname, fname) ) ENGINE = INNODB; CREATE TABLE child ( par_id INT NOT NULL, child_id INT NOT NULL, name VARCHAR(40), birth DATE, weight DECIMAL(10,2), misc_info VARCHAR(255), last_update TIMESTAMP, PRIMARY KEY (par_id, child_id), INDEX (name), FOREIGN KEY (par_id) REFERENCES parent (par_id) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE = INNODB;
Then the Table Monitor output will look something like this (reformatted slightly):
=========================================== 090420 12:09:32 INNODB TABLE MONITOR OUTPUT =========================================== -------------------------------------- TABLE: name SYS_FOREIGN, id 0 11, columns 7, indexes 3, appr.rows 1 COLUMNS: ID: DATA_VARCHAR DATA_ENGLISH len 0; FOR_NAME: DATA_VARCHAR DATA_ENGLISH len 0; REF_NAME: DATA_VARCHAR DATA_ENGLISH len 0; N_COLS: DATA_INT len 4; DB_ROW_ID: DATA_SYS prtype 256 len 6; DB_TRX_ID: DATA_SYS prtype 257 len 6; INDEX: name ID_IND, id 0 11, fields 1/6, uniq 1, type 3 root page 46, appr.key vals 1, leaf pages 1, size pages 1 FIELDS: ID DB_TRX_ID DB_ROLL_PTR FOR_NAME REF_NAME N_COLS INDEX: name FOR_IND, id 0 12, fields 1/2, uniq 2, type 0 root page 47, appr.key vals 1, leaf pages 1, size pages 1 FIELDS: FOR_NAME ID INDEX: name REF_IND, id 0 13, fields 1/2, uniq 2, type 0 root page 48, appr.key vals 1, leaf pages 1, size pages 1 FIELDS: REF_NAME ID -------------------------------------- TABLE: name SYS_FOREIGN_COLS, id 0 12, columns 7, indexes 1, appr.rows 1 COLUMNS: ID: DATA_VARCHAR DATA_ENGLISH len 0; POS: DATA_INT len 4; FOR_COL_NAME: DATA_VARCHAR DATA_ENGLISH len 0; REF_COL_NAME: DATA_VARCHAR DATA_ENGLISH len 0; DB_ROW_ID: DATA_SYS prtype 256 len 6; DB_TRX_ID: DATA_SYS prtype 257 len 6; INDEX: name ID_IND, id 0 14, fields 2/6, uniq 2, type 3 root page 49, appr.key vals 1, leaf pages 1, size pages 1 FIELDS: ID POS DB_TRX_ID DB_ROLL_PTR FOR_COL_NAME REF_COL_NAME -------------------------------------- TABLE: name test/child, id 0 14, columns 10, indexes 2, appr.rows 201 COLUMNS: par_id: DATA_INT DATA_BINARY_TYPE DATA_NOT_NULL len 4; child_id: DATA_INT DATA_BINARY_TYPE DATA_NOT_NULL len 4; name: DATA_VARCHAR prtype 524303 len 40; birth: DATA_INT DATA_BINARY_TYPE len 3; weight: DATA_FIXBINARY DATA_BINARY_TYPE len 5; misc_info: DATA_VARCHAR prtype 524303 len 255; last_update: DATA_INT DATA_UNSIGNED DATA_BINARY_TYPE DATA_NOT_NULL len 4; DB_ROW_ID: DATA_SYS prtype 256 len 6; DB_TRX_ID: DATA_SYS prtype 257 len 6; INDEX: name PRIMARY, id 0 17, fields 2/9, uniq 2, type 3 root page 52, appr.key vals 201, leaf pages 5, size pages 6 FIELDS: par_id child_id DB_TRX_ID DB_ROLL_PTR name birth weight misc_info last_update INDEX: name name, id 0 18, fields 1/3, uniq 3, type 0 root page 53, appr.key vals 210, leaf pages 1, size pages 1 FIELDS: name par_id child_id FOREIGN KEY CONSTRAINT test/child_ibfk_1: test/child ( par_id ) REFERENCES test/parent ( par_id ) -------------------------------------- TABLE: name test/innodb_table_monitor, id 0 15, columns 4, indexes 1, appr.rows 0 COLUMNS: i: DATA_INT DATA_BINARY_TYPE len 4; DB_ROW_ID: DATA_SYS prtype 256 len 6; DB_TRX_ID: DATA_SYS prtype 257 len 6; INDEX: name GEN_CLUST_INDEX, id 0 19, fields 0/4, uniq 1, type 1 root page 193, appr.key vals 0, leaf pages 1, size pages 1 FIELDS: DB_ROW_ID DB_TRX_ID DB_ROLL_PTR i -------------------------------------- TABLE: name test/parent, id 0 13, columns 6, indexes 2, appr.rows 299 COLUMNS: par_id: DATA_INT DATA_BINARY_TYPE DATA_NOT_NULL len 4; fname: DATA_CHAR prtype 524542 len 20; lname: DATA_CHAR prtype 524542 len 20; DB_ROW_ID: DATA_SYS prtype 256 len 6; DB_TRX_ID: DATA_SYS prtype 257 len 6; INDEX: name PRIMARY, id 0 15, fields 1/5, uniq 1, type 3 root page 50, appr.key vals 299, leaf pages 2, size pages 3 FIELDS: par_id DB_TRX_ID DB_ROLL_PTR fname lname INDEX: name lname, id 0 16, fields 2/3, uniq 2, type 2 root page 51, appr.key vals 300, leaf pages 1, size pages 1 FIELDS: lname fname par_id FOREIGN KEY CONSTRAINT test/child_ibfk_1: test/child ( par_id ) REFERENCES test/parent ( par_id ) ----------------------------------- END OF INNODB TABLE MONITOR OUTPUT ==================================
For each table, Table Monitor output contains a section that displays general information about the table and specific information about its columns, indexes, and foreign keys.
The general information for each table includes the table name
(in
format except for internal tables), its ID, the number of
columns and indexes, and an approximate row count.
db_name
/tbl_name
The COLUMNS
part of a table section lists
each column in the table. Information for each column
indicates its name and data type characteristics. Some
internal columns are added by InnoDB
, such
as DB_ROW_ID
(row ID),
DB_TRX_ID
(transaction ID), and
DB_ROLL_PTR
(a pointer to the rollback/undo
data).
DATA_
: These symbols indicate the data type. There may be multiplexxx
DATA_
symbols for a given column.xxx
prtype
: The column's “precise” type. This field includes information such as the column data type, character set code, nullability, signedness, and whether it is a binary string. This field is described in theinnobase/include/data0type.h
source file.len
: The column length in bytes.
Each INDEX
part of the table section
provides the name and characteristics of one table index:
name
: The index name. If the name isPRIMARY
, the index is a primary key. If the name isGEN_CLUST_INDEX
, the index is the clustered index that is created automatically if the table definition doesn't include a primary key or non-NULL
unique index. See Section 13.3.11.1, “Clustered and Secondary Indexes”.id
: The index ID.fields
: The number of fields in the index, as a value in
format:m
/n
m
is the number of user-defined columns; that is, the number of columns you would see in the index definition in aCREATE TABLE
statement.n
is the total number of index columns, including those added internally. For the clustered index, the total includes the other columns in the table definition, plus any columns added internally. For a secondary index, the total includes the columns from the primary key that are not part of the secondary index.
uniq
: The number of leading fields that are enough to determine index values uniquely.type
: The index type. This is a bit field. For example, 1 indicates a clustered index and 2 indicates a unique index, so a clustered index (which always contains unique values), will have atype
value of 3. An index with atype
value of 0 is neither clustered nor unique. The flag values are defined in theinnobase/include/dict0mem.h
source file.root page
: The index root page number.appr. key vals
: The approximate index cardinality.leaf pages
: The approximate number of leaf pages in the index.size pages
: The approximate total number of pages in the index.FIELDS
: The names of the fields in the index. For a clustered index that was generated automatically, the field list begins with the internalDB_ROW_ID
(row ID) field.DB_TRX_ID
andDB_ROLL_PTR
are always added internally to the clustered index, following the fields that comprise the primary key. For a secondary index, the final fields are those from the primary key that are not part of the secondary index.
The end of the table section lists the FOREIGN
KEY
definitions that apply to the table. This
information appears whether the table is a referencing or
referenced table.
The following general guidelines apply to troubleshooting
InnoDB
problems:
When an operation fails or you suspect a bug, look at the MySQL server error log (see Section 5.2.2, “The Error Log”).
Issues relating to the
InnoDB
data dictionary include failedCREATE TABLE
statements (orphaned table files), inability to open.InnoDB
files, and system cannot find the path specified errors. For information about these sorts of problems and errors, see Section 13.3.14.4, “TroubleshootingInnoDB
Data Dictionary Operations”.When troubleshooting, it is usually best to run the MySQL server from the command prompt, rather than through mysqld_safe or as a Windows service. You can then see what mysqld prints to the console, and so have a better grasp of what is going on. On Windows, start mysqld with the
--console
option to direct the output to the console window.Use the
InnoDB
Monitors to obtain information about a problem (see Section 13.3.14.2, “SHOW ENGINE INNODB STATUS
and theInnoDB
Monitors”). If the problem is performance-related, or your server appears to be hung, you should use the standard Monitor to print information about the internal state ofInnoDB
. If the problem is with locks, use the Lock Monitor. If the problem is in creation of tables or other data dictionary operations, use the Table Monitor to print the contents of theInnoDB
internal data dictionary. To see tablespace information use the Tablespace Monitor.If you suspect that a table is corrupt, run
CHECK TABLE
on that table.
Information about table definitions is stored both in the
.frm
files, and in the InnoDB
data dictionary. If
you move .frm
files around, or if the
server crashes in the middle of a data dictionary operation,
these sources of information can become inconsistent.
Problem with CREATE TABLE
A symptom of an out-of-sync data dictionary is that a
CREATE TABLE
statement fails. If
this occurs, look in the server's error log. If the log says
that the table already exists inside the
InnoDB
internal data dictionary, you have an
orphaned table inside the InnoDB
tablespace
files that has no corresponding .frm
file.
The error message looks like this:
InnoDB: Error: table test/parent already exists in InnoDB internal InnoDB: data dictionary. Have you deleted the .frm file InnoDB: and not used DROP TABLE? Have you used DROP DATABASE InnoDB: for InnoDB tables in MySQL version <= 3.23.43? InnoDB: See the Restrictions section of the InnoDB manual. InnoDB: You can drop the orphaned table inside InnoDB by InnoDB: creating an InnoDB table with the same name in another InnoDB: database and moving the .frm file to the current database. InnoDB: Then MySQL thinks the table exists, and DROP TABLE will InnoDB: succeed.
You can drop the orphaned table by following the instructions
given in the error message. If you are still unable to use
DROP TABLE
successfully, the
problem may be due to name completion in the
mysql client. To work around this problem,
start the mysql client with the
--skip-auto-rehash
option and try DROP TABLE
again.
(With name completion on, mysql tries to
construct a list of table names, which fails when a problem such
as just described exists.)
Problem Opening Table
Another symptom of an out-of-sync data dictionary is that MySQL
prints an error that it cannot open a
.InnoDB
file:
ERROR 1016: Can't open file: 'child2.InnoDB'. (errno: 1)
In the error log you can find a message like this:
InnoDB: Cannot find table test/child2 from the internal data dictionary InnoDB: of InnoDB though the .frm file for the table exists. Maybe you InnoDB: have deleted and recreated InnoDB data files but have forgotten InnoDB: to delete the corresponding .frm files of InnoDB tables?
This means that there is an orphaned .frm
file without a corresponding table inside
InnoDB
. You can drop the orphaned
.frm
file by deleting it manually.
Problem with Temporary Table
If MySQL crashes in the middle of an ALTER
TABLE
operation, you may end up with an orphaned
temporary table inside the InnoDB
tablespace.
Using the Table Monitor, you can see listed a table with a name
that begins with #sql-
. You can perform SQL
statements on tables whose name contains the character
“#
” if you enclose the name
within backticks. Thus, you can drop such an orphaned table like
any other orphaned table using the method described earlier. To
copy or rename a file in the Unix shell, you need to put the
file name in double quotation marks if the file name contains
“#
”.
Problem with Missing Tablespace
With innodb_file_per_table
enabled, the following message might occur if the
.frm
or .ibd
files (or
both) are missing:
InnoDB: in InnoDB data dictionary has tablespace id N
,
InnoDB: but tablespace with that id or name does not exist. Have
InnoDB: you deleted or moved .ibd files?
InnoDB: This may also be a table created with CREATE TEMPORARY TABLE
InnoDB: whose .ibd and .frm files MySQL automatically removed, but the
InnoDB: table still exists in the InnoDB internal data dictionary.
If this occurs, try the following procedure to resolve the problem:
Create a matching
.frm
file in some other database directory and copy it to the database directory where the orphan table is located.Issue
DROP TABLE
for the original table. That should successfully drop the table andInnoDB
should print a warning to the error log that the.ibd
file was missing.
Do not convert MySQL system tables in the
mysql
database from MyISAM
to InnoDB
tables! This is an unsupported
operation. If you do this, MySQL does not restart until you
restore the old system tables from a backup or re-generate them
with the mysql_install_db script.
It is not a good idea to configure InnoDB
to
use data files or log files on NFS volumes. Otherwise, the files
might be locked by other processes and become unavailable for
use by MySQL.
Maximums and Minimums
A table can contain a maximum of 1000 columns.
By default, an index key for a single-column index can be up to 767 bytes. The same length limit applies to any index key prefix. See Section 12.1.13, “
CREATE INDEX
Синтаксис”. For example, you might hit this limit with a column prefix index of more than 255 characters on aTEXT
orVARCHAR
column, assuming a UTF-8 character set and the maximum of 3 bytes for each character. When theinnodb_large_prefix
configuration option is enabled, this length limit is raised to 3072 bytes, forInnoDB
tables that use theDYNAMIC
andCOMPRESSED
row formats.When you attempt to specify an index prefix length longer than allowed, the length is silently reduced to the maximum length. This configuration option changes the error handling for some combinations of row format and prefix length longer than the maximum allowed. See
innodb_large_prefix
for details.The
InnoDB
internal maximum key length is 3500 bytes, but MySQL itself restricts this to 3072 bytes. This limit applies to the length of the combined index key in a multi-column index.The maximum row length, except for variable-length columns (
VARBINARY
,VARCHAR
,BLOB
andTEXT
), is slightly less than half of a database page. That is, the maximum row length is about 8000 bytes.LONGBLOB
andLONGTEXT
columns must be less than 4GB, and the total row length, includingBLOB
andTEXT
columns, must be less than 4GB.If a row is less than half a page long, all of it is stored locally within the page. If it exceeds half a page, variable-length columns are chosen for external off-page storage until the row fits within half a page, as described in Section 13.3.12.2, “File Space Management”.
Although
InnoDB
supports row sizes larger than 65,535 bytes internally, MySQL inself imposes a row-size limit of 65,535 for the combined size of all columns:mysql>
CREATE TABLE t (a VARCHAR(8000), b VARCHAR(10000),
->c VARCHAR(10000), d VARCHAR(10000), e VARCHAR(10000),
->f VARCHAR(10000), g VARCHAR(10000)) ENGINE=InnoDB;
ERROR 1118 (42000): Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. You have to change some columns to TEXT or BLOBsSee Section E.10.4, “Table Column-Count and Row-Size Limits”.
On some older operating systems, files must be less than 2GB. This is not a limitation of
InnoDB
itself, but if you require a large tablespace, you will need to configure it using several smaller data files rather than one or a file large data files.The combined size of the
InnoDB
log files must be less than 4GB.The minimum tablespace size is 10MB. The maximum tablespace size is four billion database pages (64TB). This is also the maximum size for a table.
The default database page size in
InnoDB
is 16KB.ЗамечаниеChanging the page size is not a supported operation and there is no guarantee that
InnoDB
will function normally with a page size other than 16KB. Problems compiling or running InnoDB may occur. In particular,ROW_FORMAT=COMPRESSED
in the Barracuda file format assumes that the page size is at most 16KB and uses 14-bit pointers.A version of
InnoDB
built for one page size cannot use data files or log files from a version built for a different page size. This limitation could affect restore or downgrade operations using data from MySQL 5.6, which does support page sizes other than 16KB.
Index Types
InnoDB
tables do not supportFULLTEXT
indexes.InnoDB
tables support spatial data types, but not indexes on them.
Restrictions on InnoDB Tables
ANALYZE TABLE
determines index cardinality (as displayed in theCardinality
column ofSHOW INDEX
output) by doing eight random dives to each of the index trees and updating index cardinality estimates accordingly. Because these are only estimates, repeated runs ofANALYZE TABLE
may produce different numbers. This makesANALYZE TABLE
fast onInnoDB
tables but not 100% accurate because it does not take all rows into account.You can change the number of random dives by modifying the
innodb_stats_sample_pages
system variable. For more information, see Section 13.4.8, “Changes for Flexibility, Ease of Use and Reliability”.MySQL uses index cardinality estimates only in join optimization. If some join is not optimized in the right way, you can try using
ANALYZE TABLE
. In the few cases thatANALYZE TABLE
does not produce values good enough for your particular tables, you can useFORCE INDEX
with your queries to force the use of a particular index, or set themax_seeks_for_key
system variable to ensure that MySQL prefers index lookups over table scans. See Section 5.1.3, “Server System Variables”, and Section C.5.6, “Optimizer-Related Issues”.SHOW TABLE STATUS
does not give accurate statistics onInnoDB
tables, except for the physical size reserved by the table. The row count is only a rough estimate used in SQL optimization.InnoDB
does not keep an internal count of rows in a table because concurrent transactions might “see” different numbers of rows at the same time. To process aSELECT COUNT(*) FROM t
statement,InnoDB
scans an index of the table, which takes some time if the index is not entirely in the buffer pool. If your table does not change often, using the MySQL query cache is a good solution. To get a fast count, you have to use a counter table you create yourself and let your application update it according to the inserts and deletes it does. If an approximate row count is sufficient,SHOW TABLE STATUS
can be used. See Section 13.3.14.1, “InnoDB
Performance Tuning Tips”.On Windows,
InnoDB
always stores database and table names internally in lowercase. To move databases in a binary format from Unix to Windows or from Windows to Unix, create all databases and tables using lowercase names.For an
AUTO_INCREMENT
column, you must always define an index for the table, and that index must contain just theAUTO_INCREMENT
column. InMyISAM
tables, theAUTO_INCREMENT
column may be part of a multiple-column index.While initializing a previously specified
AUTO_INCREMENT
column on a table,InnoDB
sets an exclusive lock on the end of the index associated with theAUTO_INCREMENT
column. While accessing the auto-increment counter,InnoDB
uses a specificAUTO-INC
table lock mode where the lock lasts only to the end of the current SQL statement, not to the end of the entire transaction. Other clients cannot insert into the table while theAUTO-INC
table lock is held. See Section 13.3.5.3, “AUTO_INCREMENT
Handling inInnoDB
”.When you restart the MySQL server,
InnoDB
may reuse an old value that was generated for anAUTO_INCREMENT
column but never stored (that is, a value that was generated during an old transaction that was rolled back).When an
AUTO_INCREMENT
integer column runs out of values, a subsequentINSERT
operation returns a duplicate-key error. This is general MySQL behavior, similar to howMyISAM
works.DELETE FROM
does not regenerate the table but instead deletes all rows, one by one.tbl_name
Under some conditions,
TRUNCATE
for antbl_name
InnoDB
table is mapped toDELETE FROM
. See Section 12.1.33, “tbl_name
TRUNCATE TABLE
Синтаксис”.Currently, cascaded foreign key actions do not activate triggers.
You cannot create a table with a column name that matches the name of an internal InnoDB column (including
DB_ROW_ID
,DB_TRX_ID
,DB_ROLL_PTR
, andDB_MIX_ID
). The server reports error 1005 and refers to error –1 in the error message. This restriction applies only to use of the names in uppercase.
Locking and Transactions
LOCK TABLES
acquires two locks on each table ifinnodb_table_locks=1
(the default). In addition to a table lock on the MySQL layer, it also acquires anInnoDB
table lock. Versions of MySQL before 4.1.2 did not acquireInnoDB
table locks; the old behavior can be selected by settinginnodb_table_locks=0
. If noInnoDB
table lock is acquired,LOCK TABLES
completes even if some records of the tables are being locked by other transactions.As of MySQL 5.5.3,
innodb_table_locks=0
has no effect for tables locked explicitly withLOCK TABLES ... WRITE
. It still has an effect for tables locked for read or write byLOCK TABLES ... WRITE
implicitly (for example, through triggers) or byLOCK TABLES ... READ
.All
InnoDB
locks held by a transaction are released when the transaction is committed or aborted. Thus, it does not make much sense to invokeLOCK TABLES
onInnoDB
tables inautocommit=1
mode because the acquiredInnoDB
table locks would be released immediately.You cannot lock additional tables in the middle of a transaction because
LOCK TABLES
performs an implicitCOMMIT
andUNLOCK TABLES
.The limit of 1023 concurrent data-modifying transactions has been raised in MySQL 5.5 and above. The limit is now 128 * 1023 concurrent transactions that generate undo records. You can remove any workarounds that require changing the proper structure of your transactions, such as committing more frequently.
- 13.4.1. Introduction to InnoDB 1.1
- 13.4.2. Fast Index Creation in the InnoDB Storage Engine
- 13.4.3. InnoDB Data Compression
- 13.4.4. InnoDB File-Format Management
- 13.4.5. How InnoDB Stores Variable-Length Columns
- 13.4.6. InnoDB
INFORMATION_SCHEMA
tables - 13.4.7. InnoDB Performance and Scalability Enhancements
- 13.4.8. Changes for Flexibility, Ease of Use and Reliability
- 13.4.9. Installing the InnoDB Storage Engine
- 13.4.10. Upgrading the InnoDB Storage Engine
- 13.4.11. Downgrading the InnoDB Storage Engine
- 13.4.12. InnoDB Storage Engine Change History
- 13.4.13. Third-Party Software
- 13.4.14. List of Parameters Changed in InnoDB 1.1 and InnoDB Plugin 1.0
InnoDB 1.1 combines the familiar reliability and performance of the InnoDB storage engine, with new performance and usability enhancements. InnoDB 1.1 includes all the features that were part of the InnoDB Plugin for MySQL 5.1, plus new features specific to MySQL 5.5 and higher.
Beginning with MySQL version 5.5, InnoDB is the default storage engine, rather than MyISAM, to promote greater data reliability and reducing the chance of corruption.
The InnoDB Storage Engine for MySQL contains several important new features:
Fast index creation: add or drop indexes without copying the data
Data compression: shrink tables, to significantly reduce storage and i/o
New row format: fully off-page storage of long
BLOB
,TEXT
, andVARCHAR
columnsFile format management: protects upward and downward compatibility
INFORMATION_SCHEMA
tables: information about compression and locking
Upward and Downward Compatibility
Note that the ability to use data compression and the new row format require the use of a new InnoDB file format called Barracuda. The previous file format, used by the built-in InnoDB in MySQL 5.1 and earlier, is now called Antelope and does not support these features, but does support the other features introduced with the InnoDB storage engine.
The InnoDB storage engine is upward compatible from standard InnoDB as
built in to, and distributed with, MySQL. Existing databases can
be used with the InnoDB Storage Engine for MySQL. The new parameter
innodb_file_format
can help
protect upward and downward compatibility between InnoDB versions
and database files, allowing users to enable or disable use of new
features that can only be used with certain versions of InnoDB.
InnoDB since version 5.0.21 has a safety feature that prevents it from opening tables that are in an unknown format. However, the system tablespace may contain references to new-format tables that confuse the built-in InnoDB in MySQL 5.1 and earlier. These references are cleared in a slow shutdown.
With previous versions of InnoDB, no error would be returned until you try to access a table that is in a format “too new” for the software. To provide early feedback, InnoDB 1.1 checks the system tablespace before startup to ensure that the file format used in the database is supported by the storage engine. See Section 13.4.4.2.1, “Compatibility Check When InnoDB Is Started” for the details.
Starting with MySQL 5.4.2, you do not need to do anything special to get or install the most up-to-date InnoDB storage engine. From that version forward, the InnoDB storage engine in the server is what was formerly known as the InnoDB Plugin. Earlier versions of MySQL required some extra build and configuration steps to get the Plugin-specific features such as fast index creation and table compression.
Report any bugs in the InnoDB storage engine using the My Oracle Support site. For general discussions about InnoDB Storage Engine for MySQL, see http://forums.mysql.com/list.php?22.
InnoDB storage engine releases are numbered with version numbers independent of MySQL release numbers. The initial release of the InnoDB storage engine is version 1.0, and it is designed to work with MySQL 5.1. Version 1.1 of the InnoDB storage engine is for MySQL 5.5 and up.
The first component of the InnoDB storage engine version number designates a major release level.
The second component corresponds to the MySQL release. The digit 0 corresponds to MySQL 5.1. The digit 1 corresponds to MySQL 5.5.
The third component indicates the specific release of the InnoDB storage engine (at a given major release level and for a specific MySQL release); only bug fixes and minor functional changes are introduced at this level.
Once you have installed the InnoDB storage engine, you can check its version number in three ways:
In the error log, it is printed during startup
SELECT * FROM information_schema.plugins;
SELECT @@innodb_version;
The InnoDB storage engine writes its version number to the error log, which can be helpful in diagnosis of errors:
091105 12:28:06 InnoDB Plugin 1.0.5 started; log sequence number 46509
Note that the PLUGIN_VERSION
column in the
table INFORMATION_SCHEMA.PLUGINS
does not
display the third component of the version number, only the first
and second components, as in 1.0.
Because InnoDB 1.1 supports the “Barracuda” file format, with new on-disk data structures within both the database and log files, pay special attention to file format compatibility with respect to the following scenarios:
Downgrading from MySQL 5.5 to the MySQL 5.1 or earlier (without the InnoDB Plugin enabled), or otherwise using earlier versions of MySQL with database files created by MySQL 5.5 and higher.
Using
mysqldump
.Using MySQL replication.
Using MySQL Enterprise Backup or InnoDB Hot Backup.
WARNING: Once you create any
tables with the Barracuda file format, take care to avoid crashes
and corruptions when using those files with an earlier version of
MySQL. It is strongly
recommended that you use a “slow shutdown”
(SET GLOBAL innodb_fast_shutdown=0
) when
stopping the MySQL server before downgrading to MySQL 5.1 or
earlier. This ensures that the log files and other system
information do not cause consistency issues or startup problems
when using a prior version of MySQL.
WARNING: If you dump a database
containing compressed tables with mysqldump
,
the dump file may contain CREATE
TABLE
statements that attempt to create compressed
tables, or those using ROW_FORMAT=DYNAMIC
in
the new database. Therefore, be sure the new database is running
the InnoDB storage engine, with the proper settings for
innodb_file_format
and
innodb_file_per_table
, if you
want to have the tables re-created as they exist in the original
database. Typically, when the mysqldump
file is
loaded, MySQL and InnoDB ignore CREATE
TABLE
options they do not recognize, and the table(s)
are created in a format used by the running server.
WARNING: If you use MySQL
replication, ensure all slaves are configured with the
InnoDB storage engine, with the same settings for
innodb_file_format
and
innodb_file_per_table
. If you do
not do so, and you create tables that require the new Barracuda
file format, replication errors may occur. If a slave MySQL server
is running an older version of MySQL, it ignores the
CREATE TABLE
options to create a
compressed table or one with
ROW_FORMAT=DYNAMIC
, and creates the table
uncompressed, with ROW_FORMAT=COMPACT
.
WARNING: Version 3.0 of
InnoDB Hot Backup does not support the new Barracuda file format. Using
InnoDB Hot Backup Version 3 to backup databases in this format causes
unpredictable behavior. MySQL Enterprise Backup, the successor product to
InnoDB Hot Backup, does support tables with the Barracuda file format.
You can also back up such databases with
mysqldump
.
- 13.4.2.1. Overview of Fast Index Creation
- 13.4.2.2. Examples of Fast Index Creation
- 13.4.2.3. Implementation Details of Fast Index Creation
- 13.4.2.4. Concurrency Considerations for Fast Index Creation
- 13.4.2.5. How Crash Recovery Works with Fast Index Creation
- 13.4.2.6. Limitations of Fast Index Creation
In MySQL 5.5 and higher, or in MySQL 5.1 with the InnoDB Plugin, creating and dropping secondary indexes does not copy the contents of the entire table, making this operation much more efficient than with prior releases.
With MySQL 5.5 and higher, or MySQL 5.1 with the InnoDB Plugin,
creating and dropping
secondary indexes for
InnoDB tables is much faster than before. Historically, adding or
dropping an index on a table with existing data could be very
slow. The CREATE INDEX
and
DROP INDEX
statements worked by
creating a new, empty table defined with the requested set of
indexes, then copying the existing rows to the new table
one-by-one, updating the indexes as the rows are inserted. After
all rows from the original table were copied, the old table was
dropped and the copy was renamed with the name of the original
table.
The performance speedup for fast index creation applies to secondary indexes, not to the primary key index. The rows of an InnoDB table are stored in a clustered index organized based on the primary key, forming what some database systems call an “index-organized table”. Because the table structure is so closely tied to the primary key, redefining the primary key still requires copying the data.
This new mechanism also means that you can generally speed the overall process of creating and loading an indexed table by creating the table with only the clustered index, and adding the secondary indexes after the data is loaded.
Although no syntax changes are required in the
CREATE INDEX
or
DROP INDEX
commands, some factors
affect the performance, space usage, and semantics of this
operation (see Section 13.4.2.6, “Limitations of Fast Index Creation”).
It is possible to create multiple indexes on a table with one
ALTER TABLE
statement. This is
relatively efficient, because the clustered index of the table
needs to be scanned only once (although the data is sorted
separately for each new index). For example:
CREATE TABLE T1(A INT PRIMARY KEY, B INT, C CHAR(1)) ENGINE=InnoDB; INSERT INTO T1 VALUES (1,2,'a'), (2,3,'b'), (3,2,'c'), (4,3,'d'), (5,2,'e'); COMMIT; ALTER TABLE T1 ADD INDEX (B), ADD UNIQUE INDEX (C);
The above statements create table T1
with the
clustered index (primary key) on column A
,
insert several rows, and then build two new indexes on columns
B
and C
. If there were many
rows inserted into T1
before the
ALTER TABLE
statement, this
approach is much more efficient than creating all the secondary
indexes before loading the data.
You can also create the indexes one at a time, but then the
clustered index of the table is scanned (as well as sorted) once
for each CREATE INDEX
statement.
Thus, the following statements are not as efficient as the
ALTER TABLE
statement above, even
though neither requires recreating the clustered index for table
T1
.
CREATE INDEX B ON T1 (B); CREATE UNIQUE INDEX C ON T1 (C);
Dropping InnoDB secondary indexes also does not require any
copying of table data. You can equally quickly drop multiple
indexes with a single ALTER TABLE
statement or multiple DROP INDEX
statements:
ALTER TABLE T1 DROP INDEX B, DROP INDEX C;
or:
DROP INDEX B ON T1; DROP INDEX C ON T1;
Restructuring the clustered index in InnoDB always requires
copying the data in the table. For example, if you create a table
without a primary key, InnoDB chooses one for you, which may be
the first UNIQUE
key defined on NOT
NULL
columns, or a system-generated key. Defining a
PRIMARY KEY
later causes the data to be copied,
as in the following example:
CREATE TABLE T2 (A INT, B INT) ENGINE=InnoDB; INSERT INTO T2 VALUES (NULL, 1); ALTER TABLE T2 ADD PRIMARY KEY (B);
When you create a UNIQUE
or PRIMARY
KEY
index, InnoDB must do some extra work. For
UNIQUE
indexes, InnoDB checks that the table
contains no duplicate values for the key. For a PRIMARY
KEY
index, InnoDB also checks that none of the
PRIMARY KEY
columns contains a
NULL
. It is best to define the primary key when
you create a table, so you need not rebuild the table later.
InnoDB has two types of indexes: the clustered index and secondary indexes. Since the clustered index contains the data values in its B-tree nodes, adding or dropping a clustered index does involve copying the data, and creating a new copy of the table. A secondary index, however, contains only the index key and the value of the primary key. This type of index can be created or dropped without copying the data in the clustered index. Because each secondary index contains copies of the primary key values (used to access the clustered index when needed), when you change the definition of the primary key, all secondary indexes are recreated as well.
Dropping a secondary index is simple. Only the internal InnoDB system tables and the MySQL data dictionary tables are updated to reflect the fact that the index no longer exists. InnoDB returns the storage used for the index to the tablespace that contained it, so that new indexes or additional table rows can use the space.
To add a secondary index to an existing table, InnoDB scans the table, and sorts the rows using memory buffers and temporary files in order by the values of the secondary index key columns. The B-tree is then built in key-value order, which is more efficient than inserting rows into an index in random order. Because the B-tree nodes are split when they fill, building the index in this way results in a higher fill-factor for the index, making it more efficient for subsequent access.
While an InnoDB secondary index is being created or dropped, the table is locked in shared mode. Any writes to the table are blocked, but the data in the table can be read. When you alter the clustered index of a table, the table is locked in exclusive mode, because the data must be copied. Thus, during the creation of a new clustered index, all operations on the table are blocked.
A CREATE INDEX
or
ALTER TABLE
statement for an InnoDB
table always waits for currently executing transactions that are
accessing the table to commit or roll back.
ALTER TABLE
statements that
redefine an InnoDB primary key wait for all
SELECT
statements that access the table to
complete, or their containing transactions to commit. No
transactions whose execution spans the creation of the index can
be accessing the table, because the original table is dropped when
the clustered index is restructured.
Once a CREATE INDEX
or
ALTER TABLE
statement that creates
an InnoDB secondary index begins executing, queries can access the
table for read access, but cannot update the table. If an
ALTER TABLE
statement is changing
the clustered index for an InnoDB table, all queries wait until
the operation completes.
A newly-created InnoDB secondary index contains only the committed
data in the table at the time the CREATE
INDEX
or ALTER TABLE
statement begins to execute. It does not contain any uncommitted
values, old versions of values, or values marked for deletion but
not yet removed from the old index.
Because a newly-created index contains only information about data current at the time the index was created, queries that need to see data that was deleted or changed before the index was created cannot use the index. The only queries that could be affected by this limitation are those executing in transactions that began before the creation of the index was begun. For such queries, unpredictable results could occur. Newer queries can use the index.
Although no data is lost if the server crashes while an
ALTER TABLE
statement is executing,
the crash recovery
process is different for
clustered indexes and
secondary indexes.
If the server crashes while creating an InnoDB secondary index,
upon recovery, MySQL drops any partially created indexes. You must
re-run the ALTER TABLE
or
CREATE INDEX
statement.
When a crash occurs during the creation of an InnoDB clustered index, recovery is more complicated, because the data in the table must be copied to an entirely new clustered index. Remember that all InnoDB tables are stored as clustered indexes. In the following discussion, we use the word table and clustered index interchangeably.
MySQL creates the new clustered index by copying the existing data from the original InnoDB table to a temporary table that has the desired index structure. Once the data is completely copied to this temporary table, the original table is renamed with a different temporary table name. The temporary table comprising the new clustered index is renamed with the name of the original table, and the original table is dropped from the database.
If a system crash occurs while creating a new clustered index, no data is lost, but you must complete the recovery process using the temporary tables that exist during the process. Since it is rare to re-create a clustered index or re-define primary keys on large tables, or to encounter a system crash during this operation, this manual does not provide information on recovering from this scenario. Instead, please see the InnoDB web site: http://www.innodb.com/support/tips.
Take the following considerations into account when creating or dropping InnoDB indexes:
During index creation, files are written to the temporary directory (
$TMPDIR
on Unix,%TEMP%
on Windows, or the value of the--tmpdir
configuration variable). Each temporary file is large enough to hold one column that makes up the new index, and each one is removed as soon as it is merged into the final index.The table is copied, rather than using Fast Index Creation when you create an index on a
TEMPORARY TABLE
. This has been reported as MySQL Bug #39833.To avoid consistency issues between the InnoDB data dictionary and the MySQL data dictionary, the table is copied, rather than using Fast Index Creation when you use the
ALTER TABLE ... RENAME COLUMN
syntax.The statement
ALTER IGNORE TABLE
does not delete duplicate rows. This has been reported as MySQL Bug #40344. Thet
ADD UNIQUE INDEXIGNORE
keyword is ignored. If any duplicate rows exist, the operation fails with the following error message:ERROR 23000: Duplicate entry '
347
' for key 'pl
'As noted above, a newly-created index contains only information about data current at the time the index was created. Therefore, you should not run queries in a transaction that might use a secondary index that did not exist at the beginning of the transaction. There is no way for InnoDB to access “old” data that is consistent with the rest of the data read by the transaction. See the discussion of locking in Section 13.4.2.4, “Concurrency Considerations for Fast Index Creation”.
Prior to InnoDB storage engine 1.0.4, unexpected results could occur if a query attempts to use an index created after the start of the transaction containing the query. If an old transaction attempts to access a “too new” index, InnoDB storage engine 1.0.4 and later reports an error:
ERROR HY000: Table definition has changed, please retry transaction
As the error message suggests, committing (or rolling back) the transaction, and restarting it, cures the problem.
InnoDB storage engine 1.0.2 introduces some improvements in error handling when users attempt to drop indexes. See section Section 13.4.8.6, “Better Error Handling when Dropping Indexes” for details.
MySQL 5.5 does not support efficient creation or dropping of
FOREIGN KEY
constraints. Therefore, if you useALTER TABLE
to add or remove aREFERENCES
constraint, the child table is copied, rather than using Fast Index Creation.
By setting InnoDB configuration options, you can create tables where the data is stored in compressed form. The compression means less data is transferred between disk and memory, and takes up less space in memory. The benefits are amplified for tables with secondary indexes, because index data is compressed also.
Because processors and cache memories have increased in speed more than disk storage devices, many workloads are I/O-bound. Data compression enables smaller database size, reduced I/O, and improved throughput, at the small cost of increased CPU utilization. Compression is especially valuable for read-intensive applications, on systems with enough RAM to keep frequently-used data in memory.
An InnoDB table created with
ROW_FORMAT=COMPRESSED
can use a smaller page
size on disk than the usual 16KB default. Smaller pages require
less I/O to read from and write to disk, which is especially
valuable for SSD devices.
The page size is specified through the
KEY_BLOCK_SIZE
parameter. The different page
size means the table must be in its own .ibd
file rather than the system tablespace, which requires enabling
the innodb_file_per_table
option. The level of
compression is the same regardless of the
KEY_BLOCK_SIZE
value. As you specify smaller
values for KEY_BLOCK_SIZE
, you get the I/O
benefits of increasingly smaller pages. But if you specify a value
that is too small, there is additional overhead to reorganize the
pages when data values cannot be compressed enough to fit multiple
rows in each page. There is a hard limit on how small
KEY_BLOCK_SIZE
can be for a table, based on the
lengths of the key columns for each of its indexes. Specify a
value that is too small, and the CREATE
TABLE
or ALTER TABLE
statement fails.
In the buffer pool, the compressed data is held in small pages,
with a page size based on the KEY_BLOCK_SIZE
value. For extracting or updating the column values, InnoDB also
creates a 16KB page in the buffer pool with the uncompressed data.
Within the buffer pool, any updates to the uncompressed page are
also re-written back to the equivalent compressed page. You might
need to size your buffer pool to accommodate the additional data
of both compressed and uncompressed pages, although the
uncompressed pages are
evicted from the buffer pool
when space is needed, and then uncompressed again on the next
access.
The default uncompressed size of InnoDB data pages is 16KB. You
can use the attributes ROW_FORMAT=COMPRESSED
,
KEY_BLOCK_SIZE
, or both in the
CREATE TABLE
and
ALTER TABLE
statements to enable
table compression. Depending on the combination of option values,
InnoDB uses a page size of 1KB, 2KB, 4KB, 8KB, or 16KB for the
.ibd
file of the table. (The actual compression
algorithm is not affected by the KEY_BLOCK_SIZE
value.)
Compression is applicable to tables, not to individual rows,
despite the option name ROW_FORMAT
.
To create a compressed table, you might use a statement like this:
CREATE TABLEname
(column1 INT PRIMARY KEY) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4
;
If you specify ROW_FORMAT=COMPRESSED
but not
KEY_BLOCK_SIZE
, the default compressed page
size of 8KB is used. If KEY_BLOCK_SIZE
is
specified, you can omit the attribute
ROW_FORMAT=COMPRESSED
.
Setting KEY_BLOCK_SIZE=16
typically does not
result in much compression, since the normal InnoDB page size is
16KB. This setting may still be useful for tables with many long
BLOB
, VARCHAR
or
TEXT
columns, because such values often do
compress well, and might therefore require fewer
“overflow” pages as described in
Section 13.4.3.4, “
Compressing BLOB, VARCHAR and TEXT Columns
”.
All indexes of a table (including the clustered index) are
compressed using the same page size, as specified in the
CREATE TABLE
or
ALTER TABLE
statement. Table
attributes such as ROW_FORMAT
and
KEY_BLOCK_SIZE
are not part of the
CREATE INDEX
syntax, and are
ignored if they are specified (although you see them in the output
of the SHOW CREATE TABLE
statement).
Compressed tables are stored in a format that previous versions
of InnoDB cannot process. To preserve downward compatibility of
database files, compression can be specified only when the
Barracuda data file format
is enabled using the configuration parameter
innodb_file_format
.
Table compression is also not available for the InnoDB system
tablespace. The system tablespace (space 0, the
ibdata*
files) may contain user data, but it
also contains internal InnoDB system information, and therefore
is never compressed. Thus, compression applies only to tables
(and indexes) stored in their own tablespaces.
To use compression, enable the
file-per-table
mode using the configuration parameter
innodb_file_per_table
and
enable the Barracuda disk file format using the parameter
innodb_file_format
. If
necessary, you can set these parameters in the MySQL option file
my.cnf
or my.ini
, or with
the SET
statement without shutting down the
MySQL server.
Specifying ROW_FORMAT=COMPRESSED
or
KEY_BLOCK_SIZE
in CREATE
TABLE
or ALTER TABLE
statements produces these warnings if the Barracuda file format
is not enabled. You can view them with the SHOW
WARNINGS
statement.
Level | Code | Message |
---|---|---|
Warning | 1478 | InnoDB: KEY_BLOCK_SIZE requires
innodb_file_per_table. |
Warning | 1478 | InnoDB: KEY_BLOCK_SIZE requires innodb_file_format=1 |
Warning | 1478 | InnoDB: ignoring
KEY_BLOCK_SIZE= |
Warning | 1478 | InnoDB: ROW_FORMAT=COMPRESSED requires
innodb_file_per_table. |
Warning | 1478 | InnoDB: assuming ROW_FORMAT=COMPACT. |
These messages are only warnings, not errors, and the table is created as if the options were not specified. When InnoDB “strict mode” (see Section 13.4.8.4, “InnoDB Strict Mode”) is enabled, InnoDB generates an error, not a warning, for these cases. In strict mode, the table is not created if the current configuration does not permit using compressed tables.
The “non-strict” behavior is intended to permit you
to import a mysqldump
file into a database
that does not support compressed tables, even if the source
database contained compressed tables. In that case, MySQL
creates the table in ROW_FORMAT=COMPACT
instead of preventing the operation.
When you import the dump file into a new database, if you want
to have the tables re-created as they exist in the original
database, ensure the server is running the InnoDB storage engine with
the proper settings for the configuration parameters
innodb_file_format
and
innodb_file_per_table
,
The attribute KEY_BLOCK_SIZE
is permitted
only when ROW_FORMAT
is specified as
COMPRESSED
or is omitted. Specifying a
KEY_BLOCK_SIZE
with any other
ROW_FORMAT
generates a warning that you can
view with SHOW WARNINGS
. However, the table
is non-compressed; the specified
KEY_BLOCK_SIZE
is ignored).
Level | Code | Message |
---|---|---|
Warning | 1478 | InnoDB: ignoring KEY_BLOCK_SIZE= |
If you are running in InnoDB strict mode, the combination of a
KEY_BLOCK_SIZE
with any
ROW_FORMAT
other than
COMPRESSED
generates an error, not a warning,
and the table is not created.
Table 13.4, “Meaning of CREATE TABLE
and
ALTER TABLE
options”
summarizes how the various options on
CREATE TABLE
and
ALTER TABLE
are handled.
Table 13.4. Meaning of CREATE TABLE
and
ALTER TABLE
options
Option | Usage | Описание |
---|---|---|
ROW_FORMAT=REDUNDANT | Storage format used prior to MySQL 5.0.3 | Less efficient than ROW_FORMAT=COMPACT ; for backward
compatibility |
ROW_FORMAT=COMPACT | Default storage format since MySQL 5.0.3 | Stores a prefix of 768 bytes of long column values in the clustered index page, with the remaining bytes stored in an overflow page |
ROW_FORMAT=DYNAMIC | Available only with
innodb_file_format=Barracuda | Store values within the clustered index page if they fit; if not, stores only a 20-byte pointer to an overflow page (no prefix) |
ROW_FORMAT=COMPRESSED | Available only with
innodb_file_format=Barracuda | Compresses the table and indexes using zlib to default compressed page
size of 8K bytes; implies
ROW_FORMAT=DYNAMIC |
KEY_BLOCK_SIZE= | Available only with
innodb_file_format=Barracuda | Specifies compressed page size of 1, 2, 4, 8 or 16K bytes; implies
ROW_FORMAT=DYNAMIC and
ROW_FORMAT=COMPRESSED |
Table 13.5, “CREATE/ALTER TABLE
Warnings and Ошибки when InnoDB
Strict Mode is OFF”
summarizes error conditions that occur with certain combinations
of configuration parameters and options on the
CREATE TABLE
or
ALTER TABLE
statements, and how
the options appear in the output of SHOW TABLE
STATUS
.
When InnoDB strict mode is OFF
, InnoDB
creates or alters the table, but may ignore certain settings, as
shown below. You can see the warning messages in the MySQL error
log. When InnoDB strict mode is ON
, these
specified combinations of options generate errors, and the table
is not created or altered. You can see the full description of
the error condition with SHOW ERRORS
. For
example:
mysql>CREATE TABLE x (id INT PRIMARY KEY, c INT)
->ENGINE=INNODB KEY_BLOCK_SIZE=33333;
ERROR 1005 (HY000): Can't create table 'test.x' (errno: 1478) mysql>SHOW ERRORS;
+-------+------+-------------------------------------------+ | Level | Code | Message | +-------+------+-------------------------------------------+ | Error | 1478 | InnoDB: invalid KEY_BLOCK_SIZE=33333. | | Error | 1005 | Can't create table 'test.x' (errno: 1478) | +-------+------+-------------------------------------------+ 2 rows in set (0.00 sec)
Table 13.5. CREATE/ALTER TABLE
Warnings and Ошибки when InnoDB
Strict Mode is OFF
Syntax | Warning or Error Condition | Resulting ROW_FORMAT , as shown in SHOW TABLE
STATUS |
---|---|---|
ROW_FORMAT=REDUNDANT | None | REDUNDANT |
ROW_FORMAT=COMPACT | None | COMPACT |
ROW_FORMAT=COMPRESSED or
ROW_FORMAT=DYNAMIC or
KEY_BLOCK_SIZE is specified | Ignored unless both
innodb_file_format =Barracuda
and
innodb_file_per_table
are enabled | COMPACT |
Invalid KEY_BLOCK_SIZE is specified (not 1, 2, 4, 8
or 16) | KEY_BLOCK_SIZE is ignored | the requested one, or COMPACT by default |
ROW_FORMAT=COMPRESSED and valid
KEY_BLOCK_SIZE are specified | None; KEY_BLOCK_SIZE specified is used, not the 8K
default | COMPRESSED |
KEY_BLOCK_SIZE is specified with
REDUNDANT , COMPACT
or DYNAMIC row format | KEY_BLOCK_SIZE is ignored | REDUNDANT , COMPACT or
DYNAMIC |
ROW_FORMAT is not one of
REDUNDANT ,
COMPACT , DYNAMIC
or COMPRESSED | Ignored if recognized by the MySQL parser. Otherwise, an error is issued. | COMPACT or N/A |
When InnoDB strict mode is ON
(innodb_strict_mode=1
), the InnoDB storage engine
rejects invalid ROW_FORMAT
or
KEY_BLOCK_SIZE
parameters. For compatibility
with earlier versions of InnoDB, strict mode is not enabled by
default; instead, InnoDB issues warnings (not errors) for
ignored invalid parameters.
Note that it is not possible to see the chosen
KEY_BLOCK_SIZE
using SHOW TABLE
STATUS
. The statement SHOW CREATE
TABLE
displays the KEY_BLOCK_SIZE
(even if it was ignored by InnoDB). The real compressed page
size inside InnoDB cannot be displayed by MySQL.
Most often, the internal optimizations in InnoDB described in Section 13.4.3.4, “ InnoDB Data Storage and Compression ” ensure that the system runs well with compressed data. However, because the efficiency of compression depends on the nature of your data, there are some factors you should consider to get best performance. You need to choose which tables to compress, and what compressed page size to use. You might also adjust the size of the buffer pool based on run-time performance characteristics, such as the amount of time the system spends compressing and uncompressing data.
When to Use Compression
In general, compression works best on tables that include a reasonable number of character string columns and where the data is read far more often than it is written. Because there are no guaranteed ways to predict whether or not compression benefits a particular situation, always test with a specific workload and data set running on a representative configuration. Consider the following factors when deciding which tables to compress.
Data Characteristics and Compression
A key determinant of the efficiency of compression in reducing the
size of data files is the nature of the data itself. Recall that
compression works by identifying repeated strings of bytes in a
block of data. Completely randomized data is the worst case.
Typical data often has repeated values, and so compresses
effectively. Character strings often compress well, whether
defined in CHAR
, VARCHAR
,
TEXT
or BLOB
columns. On the
other hand, tables containing mostly binary data (integers or
floating point numbers) or data that is previously compressed (for
example JPEG or PNG images)
may not generally compress well, significantly or at all.
You choose whether to turn on compression for each InnoDB tables.
A table and all of its indexes use the same (compressed) page
size. It might be that the primary key (clustered) index, which
contains the data for all columns of a table, compresses more
effectively than the secondary indexes. For those cases where
there are long rows, the use of compression might result in long
column values being stored “off-page”, as discussed
in Section 13.4.5.3, “Barracuda File Format: DYNAMIC
and
COMPRESSED
Row Formats”. Those overflow
pages may compress well. Given these considerations, for many
applications, some tables compress more effectively than others,
and you might find that your workload performs best only with a
subset of tables compressed.
Experimenting is the only way to determine whether or not to
compress a particular table. InnoDB compresses data in 16K chunks
corresponding to the uncompressed page size, and in addition to
user data, the page format includes some internal system data that
is not compressed. Compression utilities compress an entire stream
of data, and so may find more repeated strings across the entire
input stream than InnoDB would find in a table compressed in 16K
chunks. But you can get a sense of how compression efficiency by
using a utility that implements LZ77 compression (such as
gzip
or WinZip) on your data file.
Another way to test compression on a specific table is to copy
some data from your uncompressed table to a similar, compressed
table (having all the same indexes) and look at the size of the
resulting file. When you do so (if nothing else using compression
is running), you can examine the ratio of successful compression
operations to overall compression operations. (In the
INNODB_CMP
table, compare
COMPRESS_OPS
to
COMPRESS_OPS_OK
. See
INNODB_CMP
for more information.) If a high percentage of compression
operations complete successfully, the table might be a good
candidate for compression.
Compression and Application and Schema Design
Decide whether to compress data in your application or in the InnoDB table. It is usually not sensible to store data that is compressed by an application in an InnoDB compressed table. Further compression is extremely unlikely, and the attempt to compress just wastes CPU cycles.
Compressing in the Database
The InnoDB table compression is automatic and applies to all
columns and index values. The columns can still be tested with
operators such as LIKE
, and sort operations can
still use indexes even when the index values are compressed.
Because indexes are often a significant fraction of the total size
of a database, compression could result in significant savings in
storage, I/O or processor time. The compression and decompression
operations happen on the database server, which likely is a
powerful system that is sized to handle the expected load.
Compressing in the Application
If you compress data such as text in your application, before it is inserted into the database, You might save overhead for data that does not compress well by compressing some columns and not others. This approach uses CPU cycles for compression and uncompression on the client machine rather than the database server, which might be appropriate for a distributed application with many clients, or where the client machine has spare CPU cycles.
Hybrid Approach
Of course, it is possible to combine these approaches. For some applications, it may be appropriate to use some compressed tables and some uncompressed tables. It may be best to externally compress some data (and store it in uncompressed InnoDB tables) and allow InnoDB to compress (some of) the other tables in the application. As always, up-front design and real-life testing are valuable in reaching the right decision.
Workload Characteristics and Compression
In addition to choosing which tables to compress (and the page
size), the workload is another key determinant of performance. If
the application is dominated by reads, rather than updates, fewer
pages need to be reorganized and recompressed after the index page
runs out of room for the per-page “modification log”
that InnoDB maintains for compressed data. If the updates
predominantly change non-indexed columns or those containing
BLOB
s or large strings that happen to be stored
“off-page”, the overhead of compression may be
acceptable. If the only changes to a table are
INSERT
s that use a monotonically increasing
primary key, and there are few secondary indexes, there is little
need to reorganize and recompress index pages. Since InnoDB can
“delete-mark” and delete rows on compressed pages
“in place” by modifying uncompressed data,
DELETE
operations on a table are relatively
efficient.
For some environments, the time it takes to load data can be as important as run-time retrieval. Especially in data warehouse environments, many tables may be read-only or read-mostly. In those cases, it might or might not be acceptable to pay the price of compression in terms of increased load time, unless the resulting savings in fewer disk reads or in storage cost is significant.
Fundamentally, compression works best when the CPU time is available for compressing and uncompressing data. Thus, if your workload is I/O bound, rather than CPU-bound, you might find that compression can improve overall performance. When you test your application performance with different compression configurations, test on a platform similar to the planned configuration of the production system.
Configuration Characteristics and Compression
Reading and writing database pages from and to disk is the slowest aspect of system performance. Compression attempts to reduce I/O by using CPU time to compress and uncompress data, and is most effective when I/O is a relatively scarce resource compared to processor cycles.
This is often especially the case when running in a multi-user environment with fast, multi-core CPUs. When a page of a compressed table is in memory, InnoDB often uses an additional 16K in the buffer pool for an uncompressed copy of the page. The adaptive LRU algorithm in the InnoDB storage engine attempts to balance the use of memory between compressed and uncompressed pages to take into account whether the workload is running in an I/O-bound or CPU-bound manner. Still, a configuration with more memory dedicated to the InnoDB buffer pool tends to run better when using compressed tables than a configuration where memory is highly constrained.
Choosing the Compressed Page Size
The optimal setting of the compressed page size depends on the type and distribution of data that the table and its indexes contain. The compressed page size should always be bigger than the maximum record size, or operations may fail as noted in Section 13.4.3.4, “ Compression of B-Tree Pages ”.
Setting the compressed page size too large wastes some space, but the pages do not have to be compressed as often. If the compressed page size is set too small, inserts or updates may require time-consuming recompression, and the B-tree nodes may have to be split more frequently, leading to bigger data files and less efficient indexing.
Typically, you set the compressed page size to 8K or 4K bytes.
Given that the maximum InnoDB record size is around 8K,
KEY_BLOCK_SIZE=8
is usually a safe choice.
Monitoring Compression at Runtime
Overall application performance, CPU and I/O utilization and the size of disk files are good indicators of how effective compression is for your application.
To dig deeper into performance considerations for compressed tables, you can monitor compression performance at run time. using the Information Schema tables described in Пример 13.1, “Using the Compression Information Schema Tables”. These tables reflect the internal use of memory and the rates of compression used overall.
The INNODB_CMP
tables report information about
compression activity for each compressed page size
(KEY_BLOCK_SIZE
) in use. The information in
these tables is system-wide, and includes summary data across all
compressed tables in your database. You can use this data to help
decide whether or not to compress a table by examining these
tables when no other compressed tables are being accessed.
The key statistics to consider are the number of, and amount of
time spent performing, compression and uncompression operations.
Since InnoDB must split B-tree nodes when they are too full to
contain the compressed data following a modification, compare the
number of “successful” compression operations with
the number of such operations overall. Based on the information in
the INNODB_CMP
tables and overall application
performance and hardware resource utilization, you might make
changes in your hardware configuration, adjust the size of the
InnoDB buffer pool, choose a different page size, or select a
different set of tables to compress.
If the amount of CPU time required for compressing and uncompressing is high, changing to faster CPUs, or those with more cores, can help improve performance with the same data, application workload and set of compressed tables. Increasing the size of the InnoDB buffer pool might also help performance, so that more uncompressed pages can stay in memory, reducing the need to uncompress pages that exist in memory only in compressed form.
A large number of compression operations overall (compared to the
number of INSERT
, UPDATE
and
DELETE
operations in your application and the
size of the database) could indicate that some of your compressed
tables are being updated too heavily for effective compression. If
so, choose a larger page size, or be more selective about which
tables you compress.
If the number of “successful” compression operations
(COMPRESS_OPS_OK
) is a high percentage of the
total number of compression operations
(COMPRESS_OPS
), then the system is likely
performing well. If the ratio is low, then InnoDB is reorganizing,
recompressing, and splitting B-tree nodes more often than is
desirable. In this case, avoid compressing some tables, or
increase KEY_BLOCK_SIZE
for some of the
compressed tables. You might turn off compression for tables that
cause the number of “compression failures” in your
application to be more than 1% or 2% of the total. (Such a failure
ratio might be acceptable during a temporary operation such as a
data load).
This section describes some internal implementation details about compression in InnoDB. The information presented here may be helpful in tuning for performance, but is not necessary to know for basic use of compression.
Compression Algorithms
Some operating systems implement compression at the file system level. Files are typically divided into fixed-size blocks that are compressed into variable-size blocks, which easily leads into fragmentation. Every time something inside a block is modified, the whole block is recompressed before it is written to disk. These properties make this compression technique unsuitable for use in an update-intensive database system.
InnoDB implements compression with the help of the well-known zlib library, which implements the LZ77 compression algorithm. This compression algorithm is mature, robust, and efficient in both CPU utilization and in reduction of data size. The algorithm is “lossless”, so that the original uncompressed data can always be reconstructed from the compressed form. LZ77 compression works by finding sequences of data that are repeated within the data to be compressed. The patterns of values in your data determine how well it compresses, but typical user data often compresses by 50% or more.
Unlike compression performed by an application, or compression
features of some other database management systems, InnoDB
compression applies both to user data and to indexes. In many
cases, indexes can constitute 40-50% or more of the total database
size, so this difference is significant. When compression is
working well for a data set, the size of the InnoDB data files
(the .idb
files) is 25% to 50% of the
uncompressed size or possibly smaller. Depending on the workload,
this smaller database can in turn lead to a reduction in I/O, and
an increase in throughput, at a modest cost in terms of increased
CPU utilization.
InnoDB Data Storage and Compression
All user data in InnoDB is stored in pages comprising a B-tree index (the clustered index). In some other database systems, this type of index is called an “index-organized table”. Each row in the index node contains the values of the (user-specified or system-generated) primary key and all the other columns of the table.
Secondary indexes in InnoDB are also B-trees, containing pairs of values: the index key and a pointer to a row in the clustered index. The pointer is in fact the value of the primary key of the table, which is used to access the clustered index if columns other than the index key and primary key are required. Secondary index records must always fit on a single B-tree page.
The compression of B-tree nodes (of both clustered and secondary
indexes) is handled differently from compression of overflow pages
used to store long VARCHAR
,
BLOB
, or TEXT
columns, as
explained in the following sections.
Compression of B-Tree Pages
Because they are frequently updated, B-tree pages require special treatment. It is important to minimize the number of times B-tree nodes are split, as well as to minimize the need to uncompress and recompress their content.
One technique InnoDB uses is to maintain some system information in the B-tree node in uncompressed form, thus facilitating certain in-place updates. For example, this allows rows to be delete-marked and deleted without any compression operation.
In addition, InnoDB attempts to avoid unnecessary uncompression and recompression of index pages when they are changed. Within each B-tree page, the system keeps an uncompressed “modification log” to record changes made to the page. Updates and inserts of small records may be written to this modification log without requiring the entire page to be completely reconstructed.
When the space for the modification log runs out, InnoDB uncompresses the page, applies the changes and recompresses the page. If recompression fails, the B-tree nodes are split and the process is repeated until the update or insert succeeds.
Generally, InnoDB requires that each B-tree page can accommodate
at least two records. For compressed tables, this requirement has
been relaxed. Leaf pages of B-tree nodes (whether of the primary
key or secondary indexes) only need to accommodate one record, but
that record must fit in uncompressed form, in the per-page
modification log. Starting with InnoDB storage engine version 1.0.2, and
if InnoDB strict mode is ON
, the
InnoDB storage engine checks the maximum row size during
CREATE TABLE
or
CREATE INDEX
. If the row does not
fit, the following error message is issued: ERROR HY000:
Too big row
.
If you create a table when InnoDB strict mode is OFF, and a
subsequent INSERT
or UPDATE
statement attempts to create an index entry that does not fit in
the size of the compressed page, the operation fails with
ERROR 42000: Row size too large
. (This error
message does not name the index for which the record is too large,
or mention the length of the index record or the maximum record
size on that particular index page.) To solve this problem,
rebuild the table with ALTER TABLE
and select a larger compressed page size
(KEY_BLOCK_SIZE
), shorten any column prefix
indexes, or disable compression entirely with
ROW_FORMAT=DYNAMIC
or
ROW_FORMAT=COMPACT
.
Compressing BLOB, VARCHAR and TEXT Columns
In a clustered index, BLOB
,
VARCHAR
and TEXT
columns
that are not part of the primary key may be stored on separately
allocated (“overflow”) pages. We call these
off-page columns whose
values are stored on singly-linked lists of overflow pages.
For tables created in ROW_FORMAT=DYNAMIC
or
ROW_FORMAT=COMPRESSED
, the values of
BLOB
, TEXT
or
VARCHAR
columns may be stored fully off-page,
depending on their length and the length of the entire row. For
columns that are stored off-page, the clustered index record only
contains 20-byte pointers to the overflow pages, one per column.
Whether any columns are stored off-page depends on the page size
and the total size of the row. When the row is too long to fit
entirely within the page of the clustered index, InnoDB chooses
the longest columns for off-page storage until the row fits on the
clustered index page. As noted above, if a row does not fit by
itself on a compressed page, an error occurs.
Tables created in previous versions of InnoDB use the Antelope
file format, which supports only
ROW_FORMAT=REDUNDANT
and
ROW_FORMAT=COMPACT
. In these formats, InnoDB
stores the first 768 bytes of BLOB
,
VARCHAR
and TEXT
columns in
the clustered index record along with the primary key. The
768-byte prefix is followed by a 20-byte pointer to the overflow
pages that contain the rest of the column value.
When a table is in COMPRESSED
format, all data
written to overflow pages is compressed “as is”; that
is, InnoDB applies the zlib compression algorithm to the entire
data item. Other than the data, compressed overflow pages contain
an uncompressed header and trailer comprising a page checksum and
a link to the next overflow page, among other things. Therefore,
very significant storage savings can be obtained for longer
BLOB
, TEXT
or
VARCHAR
columns if the data is highly
compressible, as is often the case with text data (but not
previously compressed images).
The overflow pages are of the same size as other pages. A row containing ten columns stored off-page occupies ten overflow pages, even if the total length of the columns is only 8K bytes. In an uncompressed table, ten uncompressed overflow pages occupy 160K bytes. In a compressed table with an 8K page size, they occupy only 80K bytes. Thus, it is often more efficient to use compressed table format for tables with long column values.
Using a 16K compressed page size can reduce storage and I/O costs
for BLOB
, VARCHAR
or
TEXT
columns, because such data often compress
well, and might therefore require fewer “overflow”
pages, even though the B-tree nodes themselves take as many pages
as in the uncompressed form.
Compression and the InnoDB Buffer Pool
In a compressed InnoDB table, every compressed page (whether 1K, 2K, 4K or 8K) corresponds to an uncompressed page of 16K bytes. To access the data in a page, InnoDB reads the compressed page from disk if it is not already in the buffer pool, then uncompresses the page to its original 16K byte form. This section describes how InnoDB manages the buffer pool with respect to pages of compressed tables.
To minimize I/O and to reduce the need to uncompress a page, at times the buffer pool contains both the compressed and uncompressed form of a database page. To make room for other required database pages, InnoDB may “evict” from the buffer pool an uncompressed page, while leaving the compressed page in memory. Or, if a page has not been accessed in a while, the compressed form of the page may be written to disk, to free space for other data. Thus, at any given time, the buffer pool may contain both the compressed and uncompressed forms of the page, or only the compressed form of the page, or neither.
InnoDB keeps track of which pages to keep in memory and which to evict using a least-recently-used (LRU) list, so that “hot” or frequently accessed data tends to stay in memory. When compressed tables are accessed, InnoDB uses an adaptive LRU algorithm to achieve an appropriate balance of compressed and uncompressed pages in memory. This adaptive algorithm is sensitive to whether the system is running in an I/O-bound or CPU-bound manner. The goal is to avoid spending too much processing time uncompressing pages when the CPU is busy, and to avoid doing excess I/O when the CPU has spare cycles that can be used for uncompressing compressed pages (that may already be in memory). When the system is I/O-bound, the algorithm prefers to evict the uncompressed copy of a page rather than both copies, to make more room for other disk pages to become memory resident. When the system is CPU-bound, InnoDB prefers to evict both the compressed and uncompressed page, so that more memory can be used for “hot” pages and reducing the need to uncompress data in memory only in compressed form.
Compression and the InnoDB Log Files
Before a compressed page is written to a database file, InnoDB writes a copy of the page to the redo log (if it has been recompressed since the last time it was written to the database). This is done to ensure that redo logs will always be usable, even if a future version of InnoDB uses a slightly different compression algorithm. Therefore, some increase in the size of log files, or a need for more frequent checkpoints, can be expected when using compression. The amount of increase in the log file size or checkpoint frequency depends on the number of times compressed pages are modified in a way that requires reorganization and recompression.
Note that the redo log file format (and the database file format) are different from previous releases when using compression. The MySQL Enterprise Backup product does support this latest Barracuda file format for compressed InnoDB tables. The older InnoDB Hot Backup product can only back up tables using the file format Antelope, and thus does not support InnoDB tables that use compression.
As InnoDB evolves, new on-disk data structures are sometimes required to support new features. Features such as compressed tables (see Section 13.4.3, “InnoDB Data Compression”), and long variable-length columns stored off-page (see Section 13.4.5, “How InnoDB Stores Variable-Length Columns”) require data file formats that are not compatible with prior versions of InnoDB. These features both require use of the new Barracuda file format.
All other new features are compatible with the original Antelope file format and do not require the Barracuda file format.
This section discusses how to specify the file format for new InnoDB tables, compatibility of different file formats between MySQL releases,
Named File Formats. InnoDB 1.1 has the idea of a named file format and a configuration parameter to enable the use of features that require use of that format. The new file format is the Barracuda format, and the original InnoDB file format is called Antelope. Compressed tables and the new row format that stores long columns “off-page” require the use of the Barracuda file format or newer. Future versions of InnoDB may introduce a series of file formats, identified with the names of animals, in ascending alphabetic order.
The configuration parameter
innodb_file_format
controls
whether such statements as CREATE
TABLE
and ALTER TABLE
can be used to create tables that depend on support for the
Barracuda file format.
Although Oracle recommends using the Barracuda format for new tables where practical, in MySQL 5.5 the default file format is still Antelope, for maximum compatibility with replication configurations containing different MySQL releases.
The file format is a dynamic, global parameter that can be
specified in the MySQL option file (my.cnf
or
my.ini
) or changed with the SET
GLOBAL
command.
InnoDB 1.1 incorporates several checks to guard against the possible crashes and data corruptions that might occur if you run an older release of the MySQL server on InnoDB data files using a newer file format. These checks take place when the server is started, and when you first access a table. This section describes these checks, how you can control them, and error and warning conditions that might arise.
Backward Compatibility
Considerations of backward compatibility only apply when using a recent version of InnoDB (the InnoDB Plugin, or MySQL 5.5 and higher with InnoDB 1.1) alongside an older one (MySQL 5.1 or earlier, with the built-in InnoDB rather than the InnoDB Plugin). To minimize the chance of compatibility issues, you can standardize on the InnoDB Plugin for all your MySQL 5.1 and earlier database servers.
In general, a newer version of InnoDB may create a table or index that cannot safely be read or written with a prior version of InnoDB without risk of crashes, hangs, wrong results or corruptions. InnoDB 1.1 includes a mechanism to guard against these conditions, and to help preserve compatibility among database files and versions of InnoDB. This mechanism lets you take advantage of some new features of an InnoDB release (such as performance improvements and bug fixes), and still preserve the option of using your database with a prior version of InnoDB, by preventing accidental use of new features that create downward-incompatible disk files.
If a version of InnoDB supports a particular file format (whether or not that format is the default), you can query and update any table that requires that format or an earlier format. Only the creation of new tables using new features is limited based on the particular file format enabled. Conversely, if a tablespace contains a table or index that uses a file format that is not supported by the currently running software, it cannot be accessed at all, even for read access.
The only way to “downgrade” an InnoDB tablespace to
an earlier file format is to copy the data to a new table, in a
tablespace that uses the earlier format. This can be done with the
ALTER TABLE
statement, as described
in Section 13.4.4.4, “Downgrading the File Format”.
The easiest way to determine the file format of an existing InnoDB
tablespace is to examine the properties of the table it contains,
using the SHOW TABLE STATUS
command or querying
the table INFORMATION_SCHEMA.TABLES
. If the
Row_format
of the table is reported as
'Compressed'
or 'Dynamic'
,
the tablespace containing the table uses the Barracuda format.
Otherwise, it uses the prior InnoDB file format, Antelope.
Internal Details
Every InnoDB per-table tablespace (represented by a
*.ibd
file) file is labeled with a file format
identifier. The system tablespace (represented by the
ibdata
files) is tagged with the
“highest” file format in use in a group of InnoDB
database files, and this tag is checked when the files are opened.
Creating a compressed table, or a table with
ROW_FORMAT=DYNAMIC
, updates the file header for
the corresponding .ibd
file and the table type
in the InnoDB data dictionary with the identifier for the
Barracuda file format. From that point forward, the table cannot
be used with a version of InnoDB that does not support this new
file format. To protect against anomalous behavior, InnoDB version
5.0.21 and later performs a compatibility check when the table is
opened. (In many cases, the ALTER
TABLE
statement recreates a table and thus changes its
properties. The special case of adding or dropping indexes without
rebuilding the table is described in
Section 13.4.2, “Fast Index Creation in the InnoDB Storage Engine”.)
Definition of ib-file set
To avoid confusion, for the purposes of this discussion we define the term “ib-file set” to mean the set of operating system files that InnoDB manages as a unit. The ib-file set includes the following files:
The system tablespace (one or more
ibdata
files) that contain internal system information (including internal catalogs and undo information) and may include user data and indexes.Zero or more single-table tablespaces (also called “file per table” files, named
*.ibd
files).InnoDB log files; usually two,
ib_logfile0
andib_logfile1
. Used for crash recovery and in backups.
An “ib-file set” does not include the corresponding
.frm
files that contain metadata about InnoDB
tables. The .frm
files are created and managed
by MySQL, and can sometimes get out of sync with the internal
metadata in InnoDB.
Multiple tables, even from more than one database, can be stored in a single “ib-file set”. (In MySQL, a “database” is a logical collection of tables, what other systems refer to as a “schema” or “catalog”.)
To prevent possible crashes or data corruptions when InnoDB
opens an ib-file set, it checks that it can fully support the
file formats in use within the ib-file set. If the system is
restarted following a crash, or a “fast shutdown”
(i.e., innodb_fast_shutdown
is
greater than zero), there may be on-disk data structures (such
as redo or undo entries, or doublewrite pages) that are in a
“too-new” format for the current software. During
the recovery process, serious damage can be done to your data
files if these data structures are accessed. The startup check
of the file format occurs before any recovery process begins,
thereby preventing consistency issues with the new tables or
startup problems for the MySQL server.
Beginning with version InnoDB 1.0.1, the system tablespace
records an identifier or tag for the “highest” file
format used by any table in any of the tablespaces that is part
of the ib-file set. Checks against this file format tag are
controlled by the configuration parameter
innodb_file_format_check
,
which is ON
by default.
If the file format tag in the system tablespace is newer or
higher than the highest version supported by the particular
currently executing software and if
innodb_file_format_check
is
ON
, the following error is issued when the
server is started:
InnoDB: Error: the system tablespace is in a file format that this version doesn't support
You can also set
innodb_file_format
to a
file format name. Doing so prevents InnoDB from starting if the
current software does not support the file format specified. It
also sets the “high water mark” to the value you
specify. The ability to set
innodb_file_format_check
will be useful (with future releases of InnoDB) if you manually
“downgrade” all of the tables in an ib-file set (as
described in Section 13.4.11, “Downgrading the InnoDB Storage Engine”). You can then
rely on the file format check at startup if you subsequently use
an older version of InnoDB to access the ib-file set.
In some limited circumstances, you might want to start the
server and use an ib-file set that is in a “too
new” format (one that is not supported by the software
you are using). If you set the configuration parameter
innodb_file_format_check
to
OFF
, InnoDB opens the database, but issues
this warning message in the error log:
InnoDB: Warning: the system tablespace is in a file format that this version doesn't support
This is a very dangerous setting, as it permits the recovery
process to run, possibly corrupting your database if the
previous shutdown was a crash or “fast shutdown”.
You should only set
innodb_file_format_check
to OFF
if you are sure that the previous
shutdown was done with
innodb_fast_shutdown=0
, so that essentially
no recovery process occurs. In a future release, this
parameter setting may be renamed from OFF
to UNSAFE
. (However, until there are newer
releases of InnoDB that support additional file formats, even
disabling the startup checking is in fact
“safe”.)
The parameter
innodb_file_format_check
affects only what happens when a database is opened, not
subsequently. Conversely, the parameter
innodb_file_format
(which
enables a specific format) only determines whether or not a new
table can be created in the enabled format and has no effect on
whether or not a database can be opened.
The file format tag is a “high water mark”, and as
such it is increased after the server is started, if a table in
a “higher” format is created or an existing table
is accessed for read or write (assuming its format is
supported). If you access an existing table in a format higher
than the format the running software supports, the system
tablespace tag is not updated, but table-level compatibility
checking applies (and an error is issued), as described in
Section 13.4.4.2.2, “Compatibility Check When a Table Is Opened”.
Any time the high water mark is updated, the value of
innodb_file_format_check
is
updated as well, so the command SELECT
@@innodb_file_format_check;
displays the name of the
newest file format known to be used by tables in the currently
open ib-file set and supported by the currently executing
software.
To best illustrate this behavior, consider the scenario described in Table 13.6, “InnoDB Data File Compatibility and Related InnoDB Parameters”. Imagine that some future version of InnoDB supports the Cheetah format and that an ib-file set has been used with that version.
Table 13.6. InnoDB Data File Compatibility and Related InnoDB Parameters
innodb file format check | innodb file format | Highest file format used in ib-file set | Highest file format supported by InnoDB | Result |
---|---|---|---|---|
OFF | Antelope or Barracuda | Barracuda | Barracuda | Database can be opened; tables can be created which require Antelope or Barracuda file format |
OFF | Antelope or Barracuda | Cheetah | Barracuda | Database can be opened with a warning, since the database contains files in a “too new” format; tables can be created in Antelope or Barracuda file format; tables in Cheetah format cannot be accessed |
OFF | Cheetah | Barracuda | Barracuda | Database cannot be opened;
innodb_file_format
cannot be set to Cheetah |
ON | Antelope or Barracuda | Barracuda | Barracuda | Database can be opened; tables can be created in Antelope or Barracuda file format |
ON | Antelope or Barracuda | Cheetah | Barracuda | Database cannot be opened, since the database contains files in a “too new” format (Cheetah) |
ON | Cheetah | Barracuda | Barracuda | Database cannot be opened;
innodb_file_format
cannot be set to Cheetah |
When a table is first accessed, InnoDB (including some releases prior to InnoDB 1.0) checks that the file format of the tablespace in which the table is stored is fully supported. This check prevents crashes or corruptions that would otherwise occur when tables using a “too new” data structure are encountered.
All tables using any file format supported by a release can be
read or written (assuming the user has sufficient privileges).
The setting of the system configuration parameter
innodb_file_format
can prevent
creating a new table that uses specific file formats, even if
they are supported by a given release. Such a setting might be
used to preserve backward compatibility, but it does not prevent
accessing any table that uses any supported format.
As noted in Named File Formats, versions of MySQL older than 5.0.21 cannot reliably use database files created by newer versions if a new file format was used when a table was created. To prevent various error conditions or corruptions, InnoDB checks file format compatibility when it opens a file (for example, upon first access to a table). If the currently running version of InnoDB does not support the file format identified by the table type in the InnoDB data dictionary, MySQL reports the following error:
ERROR 1146 (42S02): Table 'test
.t1
' doesn't exist
InnoDB also writes a message to the error log:
InnoDB: tabletest
/t1
: unknown table type33
The table type should be equal to the tablespace flags, which contains the file format version as discussed in Section 13.4.4.3, “Identifying the File Format in Use”.
Versions of InnoDB prior to MySQL 4.1 did not include table format identifiers in the database files, and versions prior to MySQL 5.0.21 did not include a table format compatibility check. Therefore, there is no way to ensure proper operations if a table in a “too new” format is used with versions of InnoDB prior to 5.0.21.
The file format management capability in InnoDB 1.0 and higher (tablespace tagging and run-time checks) allows InnoDB to verify as soon as possible that the running version of software can properly process the tables existing in the database.
If you permit InnoDB to open a database containing files in a
format it does not support (by setting the parameter
innodb_file_format_check
to
OFF
), the table-level checking described in
this section still applies.
Users are strongly urged not to use database files that contain Barracuda file format tables with releases of InnoDB older than the MySQL 5.1 with the InnoDB Plugin. It is possible to “downgrade” such tables to the Antelope format with the procedure described in Section 13.4.4.4, “Downgrading the File Format”.
After you enable a given
innodb_file_format
, this
change applies only to newly created tables rather than existing
ones. If you do create a new table, the tablespace containing the
table is tagged with the “earliest” or
“simplest” file format that is required for the
table's features. For example, if you enable file format
Barracuda, and create a new table that is not compressed and does
not use ROW_FORMAT=DYNAMIC
, the new tablespace
that contains the table is tagged as using file format Antelope.
It is easy to identify the file format used by a given tablespace
or table. The table uses the Barracuda format if the
Row_format
reported by SHOW CREATE
TABLE
or INFORMATION_SCHEMA.TABLES
is
one of 'Compressed'
or
'Dynamic'
. (The Row_format
is a separate column; ignore the contents of the
Create_options
column, which may contain the
string ROW_FORMAT
.) If the table in a
tablespace uses neither of those features, the file uses the
format supported by prior releases of InnoDB, now called file
format Antelope. Then, the Row_format
is one of
'Redundant'
or 'Compact'
.
Internal Details
The file format identifier is written as part of the tablespace
flags (a 32-bit number) in the *.ibd
file in
the 4 bytes starting at position 54 of the file, most significant
byte first. (The first byte of the file is byte zero.) On some
systems, you can display these bytes in hexadecimal with the
command od -t x1 -j 54 -N 4
. If all bytes
are zero, the tablespace uses the Antelope file format (which is
the format used by the standard InnoDB storage engine up to
version 5.1). Otherwise, the least significant bit should be set
in the tablespace flags, and the file format identifier is written
in the bits 5 through 11. (Divide the tablespace flags by 32 and
take the remainder after dividing the integer part of the result
by 128.)
tablename
.ibd
Each InnoDB tablespace file (with a name matching
*.ibd
) is tagged with the file format used to
create its table and indexes. The way to downgrade the tablespace
is to re-create the table and its indexes. The easiest way to
recreate a table and its indexes is to use the command:
ALTER TABLEt
ROW_FORMAT=COMPACT
;
on each table that you want to downgrade. The
COMPACT
row format uses the file format
Antelope. It was introduced in MySQL 5.0.3.
The file format used by the standard built-in InnoDB in MySQL 5.1 is the Antelope format. The file format introduced with InnoDB Plugin 1.0 is the Barracuda format. Although no new features have been announced that would require additional new file formats, the InnoDB file format mechanism allows for future enhancements.
For the sake of completeness, these are the file format names that might be used for future file formats: Antelope, Barracuda, Cheetah, Dragon, Elk, Fox, Gazelle, Hornet, Impala, Jaguar, Kangaroo, Leopard, Moose, Nautilus, Ocelot, Porpoise, Quail, Rabbit, Shark, Tiger, Urchin, Viper, Whale, Xenops, Yak and Zebra. These file formats correspond to the internal identifiers 0..25.
This section discusses how certain InnoDB features, such as table
compression and off-page
storage of long columns, are controlled by the
ROW_FORMAT
clause of the CREATE
TABLE
statement. It discusses considerations for choosing
the right row format and compatibility of row formats between MySQL
releases.
The storage for rows and associated columns affects performance for queries and DML operations. As more rows fit into a single disk page, queries and index lookups can work faster, less cache memory is required in the InnoDB buffer pool, and less I/O is required to write out updated values for the numeric and short string columns.
All data in InnoDB is stored in database pages that make up a B-tree index (the clustered index organized according to the primary key columns). Table data and indexes both use this type of structure. The nodes of the index data structure contain the values of all the columns in that row (for the clustered index) or the index columns and the primary key columns (for secondary indexes).
Variable-length columns are an exception to this rule. Columns
such as BLOB
and VARCHAR
that are too long to fit on a B-tree page are stored on separately
allocated disk pages called
overflow pages. We call
such columns off-page
column. The values of these columns are stored on
singly-linked lists of overflow pages, and each such column has
its own list of one or more overflow pages. In some cases, all or
a prefix of the long column value is stored in the B-tree, to
avoid wasting storage and eliminating the need to read a separate
page.
The Barracuda file format
provides a new option (KEY_BLOCK_SIZE
) to
control how much column data is stored in the clustered index, and
how much is placed on overflow pages.
You specify the row format for a table with the
ROW_FORMAT
clause of the
CREATE TABLE
and
ALTER TABLE
statements.
When innodb_file_format
is set to
Barracuda and a table is created with
ROW_FORMAT=DYNAMIC
or
ROW_FORMAT=COMPRESSED
, long column values are
stored fully off-page, and the clustered index record contains
only a 20-byte pointer to the overflow page.
Whether any columns are stored off-page depends on the page size and the total size of the row. When the row is too long, InnoDB chooses the longest columns for off-page storage until the clustered index record fits on the B-tree page.
The DYNAMIC
row format maintains the efficiency
of storing the entire row in the index node if it fits (as do the
COMPACT
and REDUNDANT
formats), but this new format avoids the problem of filling B-tree
nodes with a large number of data bytes of long columns. The
DYNAMIC
format is based on the idea that if a
portion of a long data value is stored off-page, it is usually
most efficient to store all of the value off-page. With
DYNAMIC
format, shorter columns are likely to
remain in the B-tree node, minimizing the number of overflow pages
needed for any given row.
The COMPRESSED
row format uses similar internal
details for off-page storage as the DYNAMIC
row
format, with additional storage and performance considerations
from the table and index data being compressed and using smaller
page sizes. For full details about the
COMPRESSED
row format, see
Section 13.4.3, “InnoDB Data Compression”.
Early versions of InnoDB used an unnamed file format (now called
Antelope) for database files.
With that format, tables were defined with
ROW_FORMAT=COMPACT
(or
ROW_FORMAT=REDUNDANT
) and InnoDB stored up to
the first 768 bytes of variable-length columns (such as
BLOB
and VARCHAR
) in the
index record within the B-tree node, with the remainder stored on
the overflow pages.
To preserve compatibility with those prior versions, tables
created with the newest InnoDB use the prefix format, unless one
of ROW_FORMAT=DYNAMIC
or
ROW_FORMAT=COMPRESSED
is specified (or implied)
on the CREATE TABLE
statement.
With the Antelope file format, if the value of a column is 768
bytes or less, no overflow page is needed, and some savings in I/O
may result, since the value is in the B-tree node. This works well
for relatively short BLOB
s, but may cause
B-tree nodes to fill with data rather than key values, reducing
their efficiency. Tables with many BLOB
columns
could cause B-tree nodes to become too full of data, and contain
too few rows, making the entire index less efficient than if the
rows were shorter or if the column values were stored off-page.
The
INFORMATION_SCHEMA
is a MySQL feature that helps you monitor server activity to
diagnose capacity and performance issues. Several InnoDB-related
INFORMATION_SCHEMA
tables
(INNODB_CMP
,
INNODB_CMP_RESET
,
INNODB_CMPMEM
,
INNODB_CMPMEM_RESET
,
INNODB_TRX
,
INNODB_LOCKS
and
INNODB_LOCK_WAITS
)
contain live information about compressed InnoDB tables, the
compressed InnoDB buffer pool, all transactions currently executing
inside InnoDB, the locks that transactions hold and those that are
blocking transactions waiting for access to a resource (a table or
row).
The Information Schema tables are themselves plugins to the MySQL
server, and must be activated by INSTALL
statements. If they are installed, but the InnoDB storage engine
plugin is not installed, these tables appear to be empty.
This section describes the InnoDB-related Information Schema tables and shows some examples of their use.
Two new pairs of Information Schema tables provided by the InnoDB storage engine can give you some insight into how well compression is working overall. One pair of tables contains information about the number of compression operations and the amount of time spent performing compression. Another pair of tables contains information on the way memory is allocated for compression.
The tables
INNODB_CMP
and
INNODB_CMP_RESET
contain status information on the operations related to
compressed tables, which are covered in
Section 13.4.3, “InnoDB Data Compression”. The compressed page size
is in the column PAGE_SIZE
.
These two tables have identical contents, but reading from
INNODB_CMP_RESET
resets the statistics on compression and uncompression
operations. For example, if you archive the output of
INNODB_CMP_RESET
every 60 minutes, you see the statistics for each hourly period.
If you monitor the output of
INNODB_CMP
(making sure never to read INNODB_CMP_RESET
),
you see the cumulated statistics since InnoDB was started.
For the table definition, see
Table 19.1, “Columns of INNODB_CMP
and
INNODB_CMP_RESET
”.
The tables
INNODB_CMPMEM
and
INNODB_CMPMEM_RESET
contain status information on the compressed pages that reside
in the buffer pool. Please consult
Section 13.4.3, “InnoDB Data Compression” for further information on
compressed tables and the use of the buffer pool. The tables
INNODB_CMP
and
INNODB_CMP_RESET
should provide more useful statistics on compression.
Internal Details
The InnoDB storage engine uses a so-called “buddy allocator” system to manage memory allocated to pages of various sizes, from 1KB to 16KB. Each row of the two tables described here corresponds to a single page size.
These two tables have identical contents, but reading from
INNODB_CMPMEM_RESET
resets the statistics on relocation operations. For example, if
every 60 minutes you archived the output of
INNODB_CMPMEM_RESET
,
it would show the hourly statistics. If you never read
INNODB_CMPMEM_RESET
and monitored the output of
INNODB_CMPMEM
instead, it would show the cumulated statistics since InnoDB was
started.
For the table definition, see Table 19.2, “Columns of INNODB_CMPMEM and INNODB_CMPMEM_RESET”.
Пример 13.1. Using the Compression Information Schema Tables
The following is sample output from a database that contains
compressed tables (see Section 13.4.3, “InnoDB Data Compression”,
INNODB_CMP
,
and
INNODB_CMPMEM
).
The following table shows the contents of
INFORMATION_SCHEMA.INNODB_CMP
under light
load. The only compressed page size that the buffer pool
contains is 8K. Compressing or uncompressing pages has
consumed less than a second since the time the statistics were
reset, because the columns COMPRESS_TIME
and UNCOMPRESS_TIME
are zero.
page size | compress ops | compress ops ok | compress time | uncompress ops | uncompress time |
---|---|---|---|---|---|
1024 | 0 | 0 | 0 | 0 | 0 |
2048 | 0 | 0 | 0 | 0 | 0 |
4096 | 0 | 0 | 0 | 0 | 0 |
8192 | 1048 | 921 | 0 | 61 | 0 |
16384 | 0 | 0 | 0 | 0 | 0 |
According to
INNODB_CMPMEM
,
there are 6169 compressed 8KB pages in the buffer pool.
The following table shows the contents of
INFORMATION_SCHEMA.INNODB_CMPMEM
under
light load. We can see that some memory is unusable due to
fragmentation of the InnoDB memory allocator for compressed
pages: SUM(PAGE_SIZE*PAGES_FREE)=6784
. This
is because small memory allocation requests are fulfilled by
splitting bigger blocks, starting from the 16K blocks that are
allocated from the main buffer pool, using the buddy
allocation system. The fragmentation is this low, because some
allocated blocks have been relocated (copied) to form bigger
adjacent free blocks. This copying of
SUM(PAGE_SIZE*RELOCATION_OPS)
bytes has
consumed less than a second
(SUM(RELOCATION_TIME)=0)
.
Three InnoDB-related Information Schema tables make it easy to
monitor transactions and diagnose possible locking problems. The
three tables are
INNODB_TRX
,
INNODB_LOCKS
and
INNODB_LOCK_WAITS
.
Contains information about every transaction currently executing inside InnoDB, including whether the transaction is waiting for a lock, when the transaction started, and the particular SQL statement the transaction is executing.
For the table definition, see
Table 19.3, “INNODB_TRX
Columns”.
Each transaction in InnoDB that is waiting for another
transaction to release a lock
(INNODB_TRX.TRX_STATE='LOCK WAIT'
) is blocked
by exactly one “blocking lock request”. That
blocking lock request is for a row or table lock held by another
transaction in an incompatible mode. The waiting or blocked
transaction cannot proceed until the other transaction commits
or rolls back, thereby releasing the requested lock. For every
blocked transaction,
INNODB_LOCKS
contains one row that describes each lock the transaction has
requested, and for which it is waiting.
INNODB_LOCKS
also contains one row for each lock that is blocking another
transaction, whatever the state of the transaction that holds
the lock ('RUNNING'
, 'LOCK
WAIT'
, 'ROLLING BACK'
or
'COMMITTING'
). The lock that is blocking a
transaction is always held in a mode (read vs. write, shared vs.
exclusive) incompatible with the mode of requested lock.
For the table definition, see
Table 19.4, “INNODB_LOCKS
Columns”.
Using this table, you can tell which transactions are waiting
for a given lock, or for which lock a given transaction is
waiting. This table contains one or more rows for each
blocked transaction, indicating the lock it
has requested and the lock(s) that is (are) blocking that
request. The REQUESTED_LOCK_ID
refers to the
lock that a transaction is requesting, and the
BLOCKING_LOCK_ID
refers to the lock (held by
another transaction) that is preventing the first transaction
from proceeding. For any given blocked transaction, all rows in
INNODB_LOCK_WAITS
have the same value for REQUESTED_LOCK_ID
and
different values for BLOCKING_LOCK_ID
.
For the table definition, see
Table 19.5, “INNODB_LOCK_WAITS
Columns”.
Пример 13.2. Identifying Blocking Transactions
It is sometimes helpful to be able to identify which transaction is blocking another. You can use the Information Schema tables to find out which transaction is waiting for another, and which resource is being requested.
Suppose you have the following scenario, with three users running concurrently. Each user (or session) corresponds to a MySQL thread, and executes one transaction after another. Consider the state of the system when these users have issued the following commands, but none has yet committed its transaction:
User A:
BEGIN; SELECT a FROM t FOR UPDATE; SELECT SLEEP(100);
User B:
SELECT b FROM t FOR UPDATE;
User C:
SELECT c FROM t FOR UPDATE;
In this scenario, you may use this query to see who is waiting for whom:
SELECT r.trx_id waiting_trx_id, r.trx_mysql_thread_id waiting_thread, r.trx_query waiting_query, b.trx_id blocking_trx_id, b.trx_mysql_thread_id blocking_thread, b.trx_query blocking_query FROM information_schema.innodb_lock_waits w INNER JOIN information_schema.innodb_trx b ON b.trx_id = w.blocking_trx_id INNER JOIN information_schema.innodb_trx r ON r.trx_id = w.requesting_trx_id;
waiting trx id | waiting thread | waiting query | blocking trx id | blocking thread | blocking query |
---|---|---|---|---|---|
A4 | 6 | SELECT b FROM t FOR UPDATE | A3 | 5 | SELECT SLEEP(100) |
A5 | 7 | SELECT c FROM t FOR UPDATE | A3 | 5 | SELECT SLEEP(100) |
A5 | 7 | SELECT c FROM t FOR UPDATE | A4 | 6 | SELECT b FROM t FOR UPDATE |
In the above result, you can identify users by the “waiting query” or “blocking query”. As you can see:
User B (trx id
'A4'
, thread6
) and User C (trx id'A5'
, thread7
) are both waiting for User A (trx id'A3'
, thread5
).User C is waiting for User B as well as User A.
You can see the underlying data in the tables
INNODB_TRX
,
INNODB_LOCKS
,
and
INNODB_LOCK_WAITS
.
The following table shows some sample Contents of INFORMATION_SCHEMA.INNODB_TRX.
trx id | trx state | trx started | trx requested lock id | trx wait started | trx weight | trx mysql thread id | trx query |
---|---|---|---|---|---|---|---|
A3 | RUNNING | 2008-01-15 16:44:54 | NULL | NULL | 2 | 5 | SELECT SLEEP(100) |
A4 | LOCK WAIT | 2008-01-15 16:45:09 | A4:1:3:2 | 2008-01-15 16:45:09 | 2 | 6 | SELECT b FROM t FOR UPDATE |
A5 | LOCK WAIT | 2008-01-15 16:45:14 | A5:1:3:2 | 2008-01-15 16:45:14 | 2 | 7 | SELECT c FROM t FOR UPDATE |
The following table shows some sample contents of
INFORMATION_SCHEMA.INNODB_LOCKS
.
lock id | lock trx id | lock mode | lock type | lock table | lock index | lock space | lock page | lock rec | lock data |
---|---|---|---|---|---|---|---|---|---|
A3:1:3:2 | A3 | X | RECORD | `test`.`t` | `PRIMARY` | 1 | 3 | 2 | 0x0200 |
A4:1:3:2 | A4 | X | RECORD | `test`.`t` | `PRIMARY` | 1 | 3 | 2 | 0x0200 |
A5:1:3:2 | A5 | X | RECORD | `test`.`t` | `PRIMARY` | 1 | 3 | 2 | 0x0200 |
The following table shows some sample contents of
INFORMATION_SCHEMA.INNODB_LOCK_WAITS
.
Пример 13.3. More Complex Пример of Transaction Data in Information Schema Tables
Sometimes you would like to correlate the internal InnoDB locking information with session-level information maintained by MySQL. For example, you might like to know, for a given InnoDB transaction ID, the corresponding MySQL session ID and name of the user that may be holding a lock, and thus blocking another transaction.
The following output from the
INFORMATION_SCHEMA
tables is taken from a
somewhat loaded system.
As can be seen in the following tables, there are several transactions running.
The following INNODB_LOCKS
and
INNODB_LOCK_WAITS
tables shows that:
Transaction
77F
(executing anINSERT
) is waiting for transactions77E
,77D
and77B
to commit.Transaction
77E
(executing an INSERT) is waiting for transactions77D
and77B
to commit.Transaction
77D
(executing an INSERT) is waiting for transaction77B
to commit.Transaction
77B
(executing an INSERT) is waiting for transaction77A
to commit.Transaction
77A
is running, currently executingSELECT
.Transaction
E56
(executing anINSERT
) is waiting for transactionE55
to commit.Transaction
E55
(executing anINSERT
) is waiting for transaction19C
to commit.Transaction
19C
is running, currently executing anINSERT
.
Note that there may be an inconsistency between queries shown
in the two tables INNODB_TRX.TRX_QUERY
and
PROCESSLIST.INFO
. The current transaction
ID for a thread, and the query being executed in that
transaction, may be different in these two tables for any
given thread. See
Section 13.4.6.3.3, “Possible Inconsistency with PROCESSLIST
”
for an explanation.
The following table shows the contents of
INFORMATION_SCHEMA.PROCESSLIST
in a loaded
system.
ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO |
---|---|---|---|---|---|---|---|
384 | root | localhost | test | Query | 10 | update | insert into t2 values … |
257 | root | localhost | test | Query | 3 | update | insert into t2 values … |
130 | root | localhost | test | Query | 0 | update | insert into t2 values … |
61 | root | localhost | test | Query | 1 | update | insert into t2 values … |
8 | root | localhost | test | Query | 1 | update | insert into t2 values … |
4 | root | localhost | test | Query | 0 | preparing | SELECT * FROM processlist |
2 | root | localhost | test | Sleep | 566 |
| NULL |
The following table shows the contents of
INFORMATION_SCHEMA.INNODB_TRX
in a loaded
system.
trx id | trx state | trx started | trx requested lock id | trx wait started | trx weight | trx mysql thread id | trx query |
---|---|---|---|---|---|---|---|
77F | LOCK WAIT | 2008-01-15 13:10:16 | 77F :806 | 2008-01-15 13:10:16 | 1 | 876 | insert into t09 (D, B, C) values … |
77E | LOCK WAIT | 2008-01-15 13:10:16 | 77E :806 | 2008-01-15 13:10:16 | 1 | 875 | insert into t09 (D, B, C) values … |
77D | LOCK WAIT | 2008-01-15 13:10:16 | 77D :806 | 2008-01-15 13:10:16 | 1 | 874 | insert into t09 (D, B, C) values … |
77B | LOCK WAIT | 2008-01-15 13:10:16 | 77B :733:12:1 | 2008-01-15 13:10:16 | 4 | 873 | insert into t09 (D, B, C) values … |
77A | RUNNING | 2008-01-15 13:10:16 | NULL | NULL | 4 | 872 | select b, c from t09 where … |
E56 | LOCK WAIT | 2008-01-15 13:10:06 | E56 :743:6:2 | 2008-01-15 13:10:06 | 5 | 384 | insert into t2 values … |
E55 | LOCK WAIT | 2008-01-15 13:10:06 | E55 :743:38:2 | 2008-01-15 13:10:13 | 965 | 257 | insert into t2 values … |
19C | RUNNING | 2008-01-15 13:09:10 | NULL | NULL | 2900 | 130 | insert into t2 values … |
E15 | RUNNING | 2008-01-15 13:08:59 | NULL | NULL | 5395 | 61 | insert into t2 values … |
51D | RUNNING | 2008-01-15 13:08:47 | NULL | NULL | 9807 | 8 | insert into t2 values … |
The following table shows the contents of
INFORMATION_SCHEMA.INNODB_LOCK_WAITS
in a
loaded system
requesting trx id | requested lock id | blocking trx id | blocking lock id |
---|---|---|---|
77F | 77F :806 | 77E | 77E :806 |
77F | 77F :806 | 77D | 77D :806 |
77F | 77F :806 | 77B | 77B :806 |
77E | 77E :806 | 77D | 77D :806 |
77E | 77E :806 | 77B | 77B :806 |
77D | 77D :806 | 77B | 77B :806 |
77B | 77B :733:12:1 | 77A | 77A :733:12:1 |
E56 | E56 :743:6:2 | E55 | E55 :743:6:2 |
E55 | E55 :743:38:2 | 19C | 19C :743:38:2 |
The following table shows the contents of
INFORMATION_SCHEMA.INNODB_LOCKS
in a loaded
system.
lock id | lock trx id | lock mode | lock type | lock table | lock index | lock space | lock page | lock rec | lock data |
---|---|---|---|---|---|---|---|---|---|
77F :806 | 77F | AUTO_INC | TABLE | `test`.`t09` | NULL | NULL | NULL | NULL | NULL |
77E :806 | 77E | AUTO_INC | TABLE | `test`.`t09` | NULL | NULL | NULL | NULL | NULL |
77D :806 | 77D | AUTO_INC | TABLE | `test`.`t09` | NULL | NULL | NULL | NULL | NULL |
77B :806 | 77B | AUTO_INC | TABLE | `test`.`t09` | NULL | NULL | NULL | NULL | NULL |
77B :733:12:1 | 77B | X | RECORD | `test`.`t09` | `PRIMARY` | 733 | 12 | 1 | supremum pseudo-record |
77A :733:12:1 | 77A | X | RECORD | `test`.`t09` | `PRIMARY` | 733 | 12 | 1 | supremum pseudo-record |
E56 :743:6:2 | E56 | S | RECORD | `test`.`t2` | `PRIMARY` | 743 | 6 | 2 | 0, 0 |
E55 :743:6:2 | E55 | X | RECORD | `test`.`t2` | `PRIMARY` | 743 | 6 | 2 | 0, 0 |
E55 :743:38:2 | E55 | S | RECORD | `test`.`t2` | `PRIMARY` | 743 | 38 | 2 | 1922, 1922 |
19C :743:38:2 | 19C | X | RECORD | `test`.`t2` | `PRIMARY` | 743 | 38 | 2 | 1922, 1922 |
When a transaction updates a row in a table, or locks it with
SELECT FOR UPDATE
, InnoDB establishes a list
or queue of locks on that row. Similarly, InnoDB maintains a
list of locks on a table for table-level locks transactions
hold. If a second transaction wants to update a row or lock a
table already locked by a prior transaction in an incompatible
mode, InnoDB adds a lock request for the row to the
corresponding queue. For a lock to be acquired by a transaction,
all incompatible lock requests previously entered into the lock
queue for that row or table must be removed (the transactions
holding or requesting those locks either commit or roll back).
A transaction may have any number of lock requests for different
rows or tables. At any given time, a transaction may be
requesting a lock that is held by another transaction, in which
case it is blocked by that other transaction. The requesting
transaction must wait for the transaction that holds the
blocking lock to commit or rollback. If a transaction is not
waiting for a a lock, it is in the 'RUNNING'
state. If a transaction is waiting for a lock, it is in the
'LOCK WAIT'
state.
The table
INNODB_LOCKS
holds one or more row for each 'LOCK WAIT'
transaction, indicating the lock request(s) that is (are)
preventing its progress. This table also contains one row
describing each lock in a queue of locks pending for a given row
or table. The table
INNODB_LOCK_WAITS
shows which locks already held by a transaction are blocking
locks requested by other transactions.
The data exposed by the transaction and locking tables represent a glimpse into fast-changing data. This is not like other (user) tables, where the data only changes when application-initiated updates occur. The underlying data is internal system-managed data, and can change very quickly.
For performance reasons, and to minimize the chance of
misleading JOIN
s between the
INFORMATION_SCHEMA
tables, InnoDB collects
the required transaction and locking information into an
intermediate buffer whenever a SELECT
on any
of the tables is issued. This buffer is refreshed only if more
than 0.1 seconds has elapsed since the last time the buffer was
read. The data needed to fill the three tables is fetched
atomically and consistently and is saved in this global internal
buffer, forming a point-in-time “snapshot”. If
multiple table accesses occur within 0.1 seconds (as they almost
certainly do when MySQL processes a join among these tables),
then the same snapshot is used to satisfy the query.
A correct result is returned when you JOIN
any of these tables together in a single query, because the data
for the three tables comes from the same snapshot. Because the
buffer is not refreshed with every query of any of these tables,
if you issue separate queries against these tables within a
tenth of a second, the results are the same from query to query.
On the other hand, two separate queries of the same or different
tables issued more than a tenth of a second apart may see
different results, since the data come from different snapshots.
Because InnoDB must temporarily stall while the transaction and locking data is collected, too frequent queries of these tables can negatively impact performance as seen by other users.
As these tables contain sensitive information (at least
INNODB_LOCKS.LOCK_DATA
and
INNODB_TRX.TRX_QUERY
), for security reasons,
only the users with the PROCESS
privilege are
allowed to SELECT
from them.
As just described, while the transaction and locking data is
correct and consistent when these
INFORMATION_SCHEMA
tables are populated, the
underlying data changes so fast that similar glimpses at other,
similarly fast-changing data, may not be in sync. Thus, you
should be careful in comparing the data in the InnoDB
transaction and locking tables with that in the
MySQL table
PROCESSLIST
. The data from the
PROCESSLIST
table does not come from the same
snapshot as the data about locking and transactions. Even if you
issue a single SELECT
(JOIN
ing
INNODB_TRX
and PROCESSLIST
, for example), the content of
those tables is generally not consistent.
INNODB_TRX
may reference rows that are not present in
PROCESSLIST
or the currently executing SQL
query of a transaction, shown in
INNODB_TRX.TRX_QUERY
may be different from
the one in PROCESSLIST.INFO
. The query in
INNODB_TRX
is always consistent with the rest of
INNODB_TRX
,
INNODB_LOCKS
and
INNODB_LOCK_WAITS
when the data comes from the same snapshot.
- 13.4.7.1. Overview of InnoDB Performance
- 13.4.7.2. Faster Locking for Improved Scalability
- 13.4.7.3. Using Operating System Memory Allocators
- 13.4.7.4. Controlling InnoDB Change Buffering
- 13.4.7.5. Controlling Adaptive Hash Indexing
- 13.4.7.6. Changes Regarding Thread Concurrency
- 13.4.7.7. Changes in the Read-Ahead Algorithm
- 13.4.7.8. Multiple Background I/O Threads
- 13.4.7.9. Asynchronous I/O on Linux
- 13.4.7.10. Group Commit
- 13.4.7.11. Controlling the Master Thread I/O Rate
- 13.4.7.12. Controlling the Flushing Rate of Dirty Pages
- 13.4.7.13. Using the PAUSE Instruction in InnoDB Spin Loops
- 13.4.7.14. Control of Spin Lock Polling
- 13.4.7.15. Making Buffer Pool Scan Resistant
- 13.4.7.16. Improvements to Crash Recovery Performance
- 13.4.7.17. Integration with MySQL PERFORMANCE_SCHEMA
- 13.4.7.18. Improvements to Performance from Multiple Buffer Pools
- 13.4.7.19. Better Scalability with Multiple Rollback Segments
- 13.4.7.20. Better Scalability with Improved Purge Scheduling
- 13.4.7.21. Improved Log Sys Mutex
- 13.4.7.22. Separate Flush List Mutex
This section discusses recent InnoDB enhancements to performance and scalability, covering the performance features in InnoDB 1.1 with MySQL 5.5, and the features in the InnoDB Plugin for MySQL 5.1. This information is useful to any DBA or developer who is concerned with performance and scalability. Although some of the enhancements do not require any action on your part, knowing this information can still help you diagnose performance issues more quickly and modernize systems and applications that rely on older, inefficient behavior.
InnoDB has always been highly efficient, and includes several unique architectural elements to assure high performance and scalability. The latest InnoDB storage engine includes new features that take advantage of advances in operating systems and hardware platforms, such as multi-core processors and improved memory allocation systems. In addition, new configuration options let you better control some InnoDB internal subsystems to achieve the best performance with your workload.
Starting with MySQL 5.5 and InnoDB 1.1, the built-in InnoDB
storage engine within MySQL is upgraded to the full feature set
and performance of the former InnoDB Plugin. This change makes
these performance and scalability enhancements available to a much
wider audience than before, and eliminates the separate
installation step of the InnoDB Plugin. After learning about the
InnoDB performance features in this section, continue with
Глава 7, Optimization to learn the best practices for
overall MySQL performance, and Section 7.5, “Optimizing for InnoDB
Tables”
in particular for InnoDB tips and guidelines.
In MySQL and InnoDB, multiple threads of execution access shared data structures. InnoDB synchronizes these accesses with its own implementation of mutexes and read/write locks. InnoDB has historically protected the internal state of a read/write lock with an InnoDB mutex. On Unix and Linux platforms, the internal state of an InnoDB mutex is protected by a Pthreads mutex, as in IEEE Std 1003.1c (POSIX.1c).
On many platforms, there is a more efficient way to implement mutexes and read/write locks. Atomic operations can often be used to synchronize the actions of multiple threads more efficiently than Pthreads. Each operation to acquire or release a lock can be done in fewer CPU instructions, and thus result in less wasted time when threads are contending for access to shared data structures. This in turn means greater scalability on multi-core platforms.
InnoDB implements mutexes and read/write locks with the
built-in
functions provided by the GNU Compiler Collection (GCC) for atomic
memory access instead of using the Pthreads approach
previously used. More specifically, an InnoDB that is compiled
with GCC version 4.1.2 or later uses the atomic builtins instead
of a pthread_mutex_t
to implement InnoDB
mutexes and read/write locks.
On 32-bit Microsoft Windows, InnoDB has implemented mutexes (but not read/write locks) with hand-written assembler instructions. Beginning with Microsoft Windows 2000, functions for Interlocked Variable Access are available that are similar to the built-in functions provided by GCC. On Windows 2000 and higher, InnoDB makes use of the Interlocked functions. Unlike the old hand-written assembler code, the new implementation supports read/write locks and 64-bit platforms.
Solaris 10 introduced library functions for atomic operations, and InnoDB uses these functions by default. When MySQL is compiled on Solaris 10 with a compiler that does not support the built-in functions provided by the GNU Compiler Collection (GCC) for atomic memory access, InnoDB uses the library functions.
This change improves the scalability of InnoDB on multi-core systems. This feature is enabled out-of-the-box on the platforms where it is supported. You do not have to set any parameter or option to take advantage of the improved performance. On platforms where the GCC, Windows, or Solaris functions for atomic memory access are not available, InnoDB uses the traditional Pthreads method of implementing mutexes and read/write locks.
When MySQL starts, InnoDB writes a message to the log file
indicating whether atomic memory access is used for mutexes, for
mutexes and read/write locks, or neither. If suitable tools are
used to build InnoDB and the target CPU supports the atomic
operations required, InnoDB uses the built-in functions for
mutexing. If, in addition, the compare-and-swap operation can be
used on thread identifiers (pthread_t
), then
InnoDB uses the instructions for read-write locks as well.
Note: If you are building from source, ensure that the build process properly takes advantage of your platform capabilities.
For more information about the performance implications of locking, see Section 7.10, “Optimizing Locking Operations”.
When InnoDB was developed, the memory allocators supplied with
operating systems and run-time libraries were often lacking in
performance and scalability. At that time, there were no memory
allocator libraries tuned for multi-core CPUs. Therefore, InnoDB
implemented its own memory allocator in the mem
subsystem. This allocator is guarded by a single mutex, which may
become a bottleneck. InnoDB
also implements a wrapper interface around the system allocator
(malloc
and free
) that is
likewise guarded by a single mutex.
Today, as multi-core systems have become more widely available,
and as operating systems have matured, significant improvements
have been made in the memory allocators provided with operating
systems. New memory allocators perform better and are more
scalable than they were in the past. The leading high-performance
memory allocators include Hoard
,
libumem
, mtmalloc
,
ptmalloc
, tbbmalloc
, and
TCMalloc
. Most workloads, especially those
where memory is frequently allocated and released (such as
multi-table joins), benefit from using a more highly tuned memory
allocator as opposed to the internal, InnoDB-specific memory
allocator.
You can control whether InnoDB uses its own memory allocator or an
allocator of the operating system, by setting the value of the
system configuration parameter
innodb_use_sys_malloc
in the
MySQL option file (my.cnf
or
my.ini
). If set to ON
or
1
(the default), InnoDB uses the
malloc
and free
functions of
the underlying system rather than manage memory pools itself. This
parameter is not dynamic, and takes effect only when the system is
started. To continue to use the InnoDB memory allocator, set
innodb_use_sys_malloc
to
0
.
When the InnoDB memory allocator is disabled, InnoDB ignores the
value of the parameter
innodb_additional_mem_pool_size
.
The InnoDB memory allocator uses an additional memory pool for
satisfying allocation requests without having to fall back to
the system memory allocator. When the InnoDB memory allocator is
disabled, all such allocation requests are fulfilled by the
system memory allocator.
On Unix-like systems that use dynamic linking, replacing the
memory allocator may be as easy as making the environment
variable LD_PRELOAD
or
LD_LIBRARY_PATH
point to the dynamic library
that implements the allocator. On other systems, some relinking
may be necessary. Please refer to the documentation of the
memory allocator library of your choice.
Since InnoDB cannot track all memory use when the system memory
allocator is used
(innodb_use_sys_malloc
is
ON
), the section “BUFFER POOL AND
MEMORY” in the output of the SHOW ENGINE INNODB
STATUS
command only includes the buffer pool
statistics in the “Total memory allocated”. Any
memory allocated using the mem
subsystem or
using ut_malloc
is excluded.
For more information about the performance implications of InnoDB memory usage, see Section 7.9, “Buffering and Caching”.
When INSERT
, UPDATE
, and
DELETE
operations are done to a table, often
the values of indexed columns (particularly the values of
secondary keys) are not in sorted order, requiring substantial I/O
to bring secondary indexes up to date. InnoDB has an
insert buffer that
caches changes to secondary index entries when the relevant
page is not in the
buffer pool, thus
avoiding I/O operations by not reading in the page from the disk.
The buffered changes are merged when the page is loaded to the
buffer pool, and the updated page is later flushed to disk using
the normal mechanism. The InnoDB main thread merges buffered
changes when the server is nearly idle, and during a
slow shutdown.
Because it can result in fewer disk reads and writes, this feature is most valuable for workloads that are I/O-bound, for example applications with a high volume of DML operations such as bulk inserts.
However, the insert buffer occupies a part of the buffer pool, reducing the memory available to cache data pages. If the working set almost fits in the buffer pool, or if your tables have relatively few secondary indexes, it may be useful to disable insert buffering. If the working set entirely fits in the buffer pool, insert buffering does not impose any extra overhead, because it only applies to pages that are not in the buffer pool.
You can control the extent to which InnoDB performs insert
buffering with the system configuration parameter
innodb_change_buffering
. You can
turn on and off buffering for inserts, delete operations (when
index records are initially marked for deletion) and purge
operations (when index records are physically deleted). An update
operation is represented as a combination of an insert and a
delete. In MySQL 5.5 and higher, the default value is changed from
inserts
to all
.
The allowed values of
innodb_change_buffering
are:
all
The default value: buffer inserts, delete-marking operations, and purges.
none
Do not buffer any operations.
inserts
Buffer insert operations.
deletes
Buffer delete-marking operations.
changes
Buffer both inserts and delete-marking.
purges
Buffer the physical deletion operations that happen in the background.
You can set the value of this parameter in the MySQL option file
(my.cnf
or my.ini
) or change
it dynamically with the SET GLOBAL
command,
which requires the SUPER
privilege. Changing
the setting affects the buffering of new operations; the merging
of already buffered entries is not affected.
For more information about speeding up INSERT
,
UPDATE
, and DELETE
statements, see Section 7.2.2, “Optimizing DML Statements”.
If a table fits almost entirely in main memory, the fastest way to perform queries on it is to use hash indexes rather than B-tree lookups. MySQL monitors searches on each index defined for an InnoDB table. If it notices that certain index values are being accessed frequently, it automatically builds an in-memory hash table for that index. Based on the observed pattern of searches, it builds a hash index using a prefix of the index key. The prefix of the key can be any length, and it may be that only some of the values in the B-tree appear in the hash index. Hash indexes are built on demand for those pages of the index that are often accessed.
This adaptive hash index mechanism allows InnoDB to take advantage of large amounts of memory, something typically done only by database systems specifically designed for databases that reside entirely in memory. Normally, the automatic building and use of adaptive hash indexes improves performance. However, sometimes, the read/write lock that guards access to the adaptive hash index may become a source of contention under heavy workloads, such as multiple concurrent joins.
You can monitor the use of the adaptive hash index and the
contention for its use in the “SEMAPHORES” section of
the output of the SHOW ENGINE INNODB STATUS
command. If you see many threads waiting on an RW-latch created in
btr0sea.c
, then it might be useful to disable
adaptive hash indexing.
The configuration parameter
innodb_adaptive_hash_index
can be set to disable or enable the adaptive hash index. See
Section 13.4.8.2.4, “Dynamically Changing innodb_adaptive_hash_index
”
for details.
For more information about the performance characteristics of hash indexes, see Section 7.3.8, “Comparison of B-Tree and Hash Indexes”.
InnoDB uses operating system threads to process requests from user transactions. (Transactions may issue many requests to InnoDB before they commit or roll back.) On modern operating systems and servers with multi-core processors, where context switching is efficient, most workloads run well without any limit on the number of concurrent threads. Scalability improvements in MySQL 5.5 and up reduce the need to limit the number of concurrently executing threads inside InnoDB.
In situations where it is helpful to minimize context switching between threads, InnoDB can use a number of techniques to limit the number of concurrently executing operating system threads (and thus the number of requests that are processed at any one time). When InnoDB receives a new request from a user session, if the number of threads concurrently executing is at a pre-defined limit, the new request sleeps for a short time before it tries again. A request that cannot be rescheduled after the sleep is put in a first-in/first-out queue and eventually is processed. Threads waiting for locks are not counted in the number of concurrently executing threads.
You can limit the number of concurrent threads by setting the
configuration parameter
innodb_thread_concurrency
.
Once the number of executing threads reaches this limit,
additional threads sleep for a number of microseconds, set by the
configuration parameter
innodb_thread_sleep_delay
,
before being placed into the queue.
The default value for
innodb_thread_concurrency
and the implied default limit on the number of concurrent threads
has been changed in various releases of MySQL and InnoDB.
Currently, the default value of
innodb_thread_concurrency
is
0
, so that by default there is no limit on the
number of concurrently executing threads, as shown in
Table 13.7, “Changes to innodb_thread_concurrency
”.
Table 13.7. Changes to innodb_thread_concurrency
InnoDB Version | MySQL Version | Default value | Default limit of concurrent threads | Value to allow unlimited threads |
---|---|---|---|---|
Built-in | Earlier than 5.1.11 | 20 | No limit | 20 or higher |
Built-in | 5.1.11 and newer | 8 | 8 | 0 |
InnoDB before 1.0.3 | (corresponding to Plugin) | 8 | 8 | 0 |
InnoDB 1.0.3 and newer | (corresponding to Plugin) | 0 | No limit | 0 |
Note that InnoDB causes threads to sleep only when the number of
concurrent threads is limited. When there is no limit on the
number of threads, all contend equally to be scheduled. That is,
if innodb_thread_concurrency
is
0
, the value of
innodb_thread_sleep_delay
is
ignored.
When there is a limit on the number of threads, InnoDB reduces
context switching overhead by permitting multiple requests made
during the execution of a single SQL statement to enter InnoDB
without observing the limit set by
innodb_thread_concurrency
.
Since an SQL statement (such as a join) may comprise multiple row
operations within InnoDB, InnoDB assigns “tickets”
that allow a thread to be scheduled repeatedly with minimal
overhead.
When a new SQL statement starts, a thread has no tickets, and it
must observe
innodb_thread_concurrency
.
Once the thread is entitled to enter InnoDB, it is assigned a
number of tickets that it can use for subsequently entering
InnoDB. If the tickets run out,
innodb_thread_concurrency
is observed again and further tickets are assigned. The number of
tickets to assign is specified by the global option
innodb_concurrency_tickets
,
which is 500 by default. A thread that is waiting for a lock is
given one ticket once the lock becomes available.
The correct values of these variables depend on your environment and workload. Try a range of different values to determine what value works for your applications. Before limiting the number of concurrently executing threads, review configuration options that may improve the performance of InnoDB on multi-core and multi-processor computers, such as innodb_use_sys_malloc and innodb_adaptive_hash_index.
For general performance information about MySQL thread handling, see Section 7.11.5.1, “How MySQL Uses Threads for Client Connections”.
A read-ahead request is an I/O request to prefetch multiple pages in the buffer pool asynchronously, in anticipation that these pages will be needed soon. InnoDB uses or has used two read-ahead algorithms to improve I/O performance:
Linear read-ahead is based on
the access pattern of the pages in the buffer pool, not just their
number. You can control when InnoDB performs a read-ahead
operation by adjusting the number of sequential page accesses
required to trigger an asynchronous read request, using the
configuration parameter
innodb_read_ahead_threshold
.
Before this parameter was added, InnoDB would only calculate
whether to issue an asynchronous prefetch request for the entire
next extent when it read in the last page of the current extent.
Random read-ahead is a former technique that has now been removed as of MySQL 5.5. If a certain number of pages from the same extent (64 consecutive pages) were found in the buffer pool, InnoDB asynchronously issued a request to prefetch the remaining pages of the extent. Random read-ahead added unnecessary complexity to the InnoDB code and often resulted in performance degradation rather than improvement. This feature is no longer part of InnoDB, and users should generally see equivalent or improved performance.
If the number of pages read from an extent of 64 pages is greater
or equal to
innodb_read_ahead_threshold
,
InnoDB initiates an asynchronous read-ahead operation of the
entire following extent. Thus, this parameter controls how
sensitive InnoDB is to the pattern of page accesses within an
extent in deciding whether to read the following extent
asynchronously. The higher the value, the more strict the access
pattern check. For example, if you set the value to 48, InnoDB
triggers a linear read-ahead request only when 48 pages in the
current extent have been accessed sequentially. If the value is 8,
InnoDB would trigger an asynchronous read-ahead even if as few as
8 pages in the extent were accessed sequentially.
The new configuration parameter
innodb_read_ahead_threshold
can be set to any value from 0-64. The default value is 56,
meaning that an asynchronous read-ahead is performed only when 56
of the 64 pages in the extent are accessed sequentially. You can
set the value of this parameter in the MySQL option file (my.cnf
or my.ini), or change it dynamically with the SET
GLOBAL
command, which requires the
SUPER
privilege.
The SHOW ENGINE INNODB STATUS
command displays
statistics to help you evaluate the effectiveness of the
read-ahead algorithm. See
Section 13.4.8.8, “More Read-Ahead Statistics” for more
information.
For more information about I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O” and
Section 7.11.3, “Optimizing Disk I/O”.
InnoDB uses background threads
to service various types of I/O requests. You can configure the
number of background threads that service read and write I/O on
data pages, using the configuration parameters
innodb_read_io_threads
and
innodb_write_io_threads
. These
parameters signify the number of background threads used for read
and write requests respectively. They are effective on all
supported platforms. You can set the value of these parameters in
the MySQL option file (my.cnf
or
my.ini
); you cannot change them dynamically.
The default value for these parameters is 4
and
the permissible values range from 1-64
.
These parameters replace
innodb_file_io_threads
from
earlier versions of MySQL. If you try to set a value for this
obsolete parameter, a warning is written to the log file and the
value is ignored. This parameter only applied to Windows
platforms. (On non-Windows platforms, there was only one thread
each for read and write.)
The purpose of this change is to make InnoDB more scalable on high
end systems. Each background thread can handle up to 256 pending
I/O requests. A major source of background I/O is
theread-ahead
requests. InnoDB tries to balance the load of incoming requests in
such way that most of the background threads share work equally.
InnoDB also attempts to allocate read requests from the same
extent to the same thread to increase the chances of coalescing
the requests together. If you have a high end I/O subsystem and
you see more than 64 ×
innodb_read_io_threads
pending
read requests in SHOW ENGINE INNODB STATUS
, you
might gain by increasing the value of
innodb_read_io_threads
.
For more information about InnoDB I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O”.
Starting in InnoDB 1.1 with MySQL 5.5, the
asynchronous I/O
capability that InnoDB has had on Windows systems is now available
on Linux systems. (Other Unix-like systems continue to use
synchronous I/O calls.) This feature improves the scalability of
heavily I/O-bound systems, which typically show many pending
reads/writes in the output of the command SHOW ENGINE
INNODB STATUS\G
.
Running with a large number of InnoDB
I/O
threads, and especially running multiple such instances on the
same server machine, can exceed capacity limits on Linux systems.
In this case, you can fix the error:
EAGAIN: The specified maxevents exceeds the user's limit of available events.
by writing a higher limit to
/proc/sys/fs/aio-max-nr
.
In general, if a problem with the asynchronous I/O subsystem in
the OS prevents InnoDB from starting, set the option
innodb_use_native_aio=0
in the
configuration file. This new configuration option applies to Linux
systems only, and cannot be changed once the server is running.
For more information about InnoDB I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O”.
InnoDB, like any other ACID-compliant database engine, flushes the redo log of a transaction before it is committed. Historically, InnoDB used group commit functionality to group multiple such flush requests together to avoid one flush for each commit. With group commit, InnoDB issues a single write to the log file to perform the commit action for multiple user transactions that commit at about the same time, significantly improving throughput.
Group commit in InnoDB worked until MySQL 4.x, and works once again with MySQL 5.1 with the InnoDB Plugin, and MySQL 5.5 and higher. The introduction of support for the distributed transactions and Two Phase Commit (2PC) in MySQL 5.0 interfered with the InnoDB group commit functionality. This issue is now resolved.
The group commit functionality inside InnoDB works with the Two
Phase Commit protocol in MySQL. Re-enabling of the group commit
functionality fully ensures that the ordering of commit in the
MySQL binlog and the InnoDB logfile is the same as it was before.
It means it is totally safe to use MySQL Enterprise Backup
with InnoDB 1.0.4 (that is, the InnoDB Plugin with
MySQL 5.1) and above. When the binlog is enabled, you typically
also set the configuration option
sync_binlog=0
, because group commit for the
binary log is only supported if it is set to 0.
Group commit is transparent; you do not need to do anything to take advantage of this significant performance improvement.
For more information about performance of
COMMIT
and other transactional operations, see
Section 7.5.2, “Optimizing InnoDB
Transaction Management”.
The master thread in InnoDB is a thread that performs various tasks in the background. Most of these tasks are I/O related, such as flushing dirty pages from the buffer pool or writing changes from the insert buffer to the appropriate secondary indexes. The master thread attempts to perform these tasks in a way that does not adversely affect the normal working of the server. It tries to estimate the free I/O bandwidth available and tune its activities to take advantage of this free capacity. Historically, InnoDB has used a hard coded value of 100 IOPs (input/output operations per second) as the total I/O capacity of the server.
The parameter
innodb_io_capacity
indicates the overall I/O capacity available to InnoDB. This
parameter should be set to approximately the number of I/O
operations that the system can perform per second. The value
depends on your system configuration. When
innodb_io_capacity
is set,
the master threads estimates the I/O bandwidth available for
background tasks based on the set value. Setting the value to
100
reverts to the old behavior.
You can set the value of
innodb_io_capacity
to any
number 100 or greater. The default value is
200
, reflecting that the performance of typical
modern I/O devices is higher than in the early days of MySQL.
Typically, values around the previous default of 100 are
appropriate for consumer-level storage devices, such as hard
drives up to 7200 RPMs. Faster hard drives, RAID configurations,
and SSDs benefit from higher values.
You can set the value of this parameter in the MySQL option file
(my.cnf
or my.ini
) or change
it dynamically with the SET GLOBAL
command,
which requires the SUPER
privilege.
For more information about InnoDB I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O”.
InnoDB performs certain tasks in the background, including
flushing of
dirty pages (those
pages that have been changed but are not yet written to the
database files) from the
buffer pool, a task
performed by the master
thread. Currently, InnoDB aggressively flushes buffer pool
pages if the percentage of dirty pages in the buffer pool exceeds
innodb_max_dirty_pages_pct
.
InnoDB uses a new algorithm to estimate the required rate of flushing, based on the speed of redo log generation and the current rate of flushing. The intent is to smooth overall performance by ensuring that buffer flush activity keeps up with the need to keep the buffer pool “clean”. Automatically adjusting the rate of flushing can help to avoid steep dips in throughput, when excessive buffer pool flushing limits the I/O capacity available for ordinary read and write activity.
InnoDB uses its log files in a circular fashion. Before reusing a
portion of a log file, InnoDB flushes to disk all dirty buffer
pool pages whose redo entries are contained in that portion of the
log file, a process known as a
sharp checkpoint. If
a workload is write-intensive, it generates a lot of redo
information, all written to the log file. If all available space
in the log files is used up, a sharp checkpoint occurs, causing a
temporary reduction in throughput. This situation can happen even
though innodb_max_dirty_pages_pct
is not reached.
InnoDB uses a heuristic-based algorithm to avoid such a scenario, by measuring the number of dirty pages in the buffer pool and the rate at which redo is being generated. Based on these numbers, InnoDB decides how many dirty pages to flush from the buffer pool each second. This self-adapting algorithm is able to deal with sudden changes in the workload.
Internal benchmarking has also shown that this algorithm not only maintains throughput over time, but can also improve overall throughput significantly.
Because adaptive flushing is a new feature that can significantly
affect the I/O pattern of a workload, a new configuration
parameter lets you turn off this feature. The default value of the
boolean parameter
innodb_adaptive_flushing
is
TRUE
, enabling the new algorithm. You can set
the value of this parameter in the MySQL option file
(my.cnf
or my.ini
) or change
it dynamically with the SET GLOBAL
command,
which requires the SUPER
privilege.
For more information about InnoDB I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O”.
Synchronization inside InnoDB frequently involves the use of
spin loops: while waiting,
InnoDB executes a tight loop of instructions repeatedly to avoid
having the InnoDB process and
threads be rescheduled by the
operating system. If the spin loops are executed too quickly,
system resources are wasted, imposing a performance penalty on
transaction throughput. Most modern processors implement the
PAUSE
instruction for use in spin loops, so the
processor can be more efficient.
InnoDB uses a PAUSE
instruction in its spin
loops on all platforms where such an instruction is available.
This technique increases overall performance with CPU-bound
workloads, and has the added benefit of minimizing power
consumption during the execution of the spin loops.
You do not have to do anything to take advantage of this performance improvement.
For performance considerations for InnoDB locking operations, see Section 7.10, “Optimizing Locking Operations”.
Many InnoDB mutexes and rw-locks are reserved for a short time. On a multi-core system, it can be more efficient for a thread to continuously check if it can acquire a mutex or rw-lock for a while before sleeping. If the mutex or rw-lock becomes available during this polling period, the thread can continue immediately, in the same time slice. However, too-frequent polling by multiple threads of a shared object can cause “cache ping pong”, different processors invalidating portions of each others' cache. InnoDB minimizes this issue by waiting a random time between subsequent polls. The delay is implemented as a busy loop.
You can control the maximum delay between testing a mutex or
rw-lock using the parameter
innodb_spin_wait_delay
. The
duration of the delay loop depends on the C compiler and the
target processor. (In the 100MHz Pentium era, the unit of delay
was one microsecond.) On a system where all processor cores share
a fast cache memory, you might reduce the maximum delay or disable
the busy loop altogether by setting
innodb_spin_wait_delay=0
. On a system with
multiple processor chips, the effect of cache invalidation can be
more significant and you might increase the maximum delay.
The default value of
innodb_spin_wait_delay
is
6
. The spin wait delay is a dynamic global
parameter that you can specify in the MySQL option file
(my.cnf
or my.ini
) or change
at runtime with the command SET GLOBAL
innodb_spin_wait_delay=
,
where delay
is the
desired maximum delay. Changing the setting requires the
delay
SUPER
privilege.
For performance considerations for InnoDB locking operations, see Section 7.10, “Optimizing Locking Operations”.
Rather than using a strictly LRU algorithm, InnoDB uses a technique to minimize the amount of data that is brought into the buffer pool and never accessed again. The goal is to make sure that frequently accessed (“hot”) pages remain in the buffer pool, even as read-ahead and full table scans bring in new blocks that might or might not be accessed afterward.
Newly read blocks are inserted into the middle of the list
representing the buffer pool. of the LRU list. All newly read
pages are inserted at a location that by default is
3/8
from the tail of the LRU list. The pages
are moved to the front of the list (the most-recently used end)
when they are accessed in the buffer pool for the first time. Thus
pages that are never accessed never make it to the front portion
of the LRU list, and “age out” sooner than with a
strict LRU approach. This arrangement divides the LRU list into
two segments, where the pages downstream of the insertion point
are considered “old” and are desirable victims for
LRU eviction.
For an explanation of the inner workings of the InnoDB buffer pool
and the specifics of its LRU replacement algorithm, see
Section 7.9.1, “The InnoDB
Buffer Pool”.
You can control the insertion point in the LRU list, and choose
whether InnoDB applies the same optimization to blocks brought
into the buffer pool by table or index scans. The configuration
parameter
innodb_old_blocks_pct
controls the percentage of “old” blocks in the LRU
list. The default value of
innodb_old_blocks_pct
is
37
, corresponding to the original fixed ratio
of 3/8. The value range is 5
(new pages in the
buffer pool age out very quickly) to 95
(only
5% of the buffer pool is reserved for hot pages, making the
algorithm close to the familiar LRU strategy).
The optimization that keeps the buffer pool from being churned by
read-ahead can avoid similar problems due to table or index scans.
In these scans, a data page is typically accessed a few times in
quick succession and is never touched again. The configuration
parameter innodb_old_blocks_time
specifies the time window (in milliseconds) after the first access
to a page during which it can be accessed without being moved to
the front (most-recently used end) of the LRU list. The default
value of innodb_old_blocks_time
is 0
, corresponding to the original behavior of
moving a page to the most-recently used end of the buffer pool
list when it is first accessed in the buffer pool. Increasing this
value makes more and more blocks likely to age out faster from the
buffer pool.
Both the new parameters
innodb_old_blocks_pct
and
innodb_old_blocks_time
are
dynamic, global and can be specified in the MySQL option file
(my.cnf
or my.ini
) or
changed at runtime with the SET GLOBAL
command.
Changing the setting requires the SUPER
privilege.
To help you gauge the effect of setting these parameters, the
SHOW ENGINE INNODB STATUS
command reports
additional statistics. The BUFFER POOL AND
MEMORY
section now looks like:
Total memory allocated 1107296256; in additional pool allocated 0 Dictionary memory allocated 80360 Buffer pool size 65535 Free buffers 0 Database pages 63920 Old database pages 23600 Modified db pages 34969 Pending reads 32 Pending writes: LRU 0, flush list 0, single page 0 Pages made young 414946, not young 2930673 1274.75 youngs/s, 16521.90 non-youngs/s Pages read 486005, created 3178, written 160585 2132.37 reads/s, 3.40 creates/s, 323.74 writes/s Buffer pool hit rate 950 / 1000, young-making rate 30 / 1000 not 392 / 1000 Pages read ahead 1510.10/s, evicted without access 0.00/s LRU len: 63920, unzip_LRU len: 0 I/O sum[43690]:cur[221], unzip sum[0]:cur[0]
Old database pages
is the number of pages in the “old” segment of the LRU list.Pages made young
andnot young
is the total number of “old” pages that have been made young or not respectively.youngs/s
andnon-young/s
is the rate at which page accesses to the “old” pages have resulted in making such pages young or otherwise respectively since the last invocation of the command.young-making rate
andnot
provides the same rate but in terms of overall buffer pool accesses instead of accesses just to the “old” pages.
Because the effects of these parameters can vary widely based on your hardware configuration, your data, and the details of your workload, always benchmark to verify the effectiveness before changing these settings in any performance-critical or production environment.
In mixed workloads where most of the activity is OLTP type with
periodic batch reporting queries which result in large scans,
setting the value of
innodb_old_blocks_time
during the batch runs can help keep the working set of the normal
workload in the buffer pool.
When scanning large tables that cannot fit entirely in the buffer
pool, setting
innodb_old_blocks_pct
to a
small value keeps the data that is only read once from consuming a
significant portion of the buffer pool. For example, setting
innodb_old_blocks_pct=5
restricts this data
that is only read once to 5% of the buffer pool.
When scanning small tables that do fit into memory, there is less
overhead for moving pages around within the buffer pool, so you
can leave innodb_old_blocks_pct
at its default value, or even higher, such as
innodb_old_blocks_pct=50
.
The effect of the
innodb_old_blocks_time
parameter is harder to predict than the
innodb_old_blocks_pct
parameter, is relatively small, and varies more with the workload.
To arrive at an optimal value, conduct your own benchmarks if the
performance improvement from adjusting
innodb_old_blocks_pct
is
not sufficient.
For more information about the InnoDB buffer pool, see
Section 7.9.1, “The InnoDB
Buffer Pool”.
A number of optimizations speed up certain steps of the recovery that happens on the next startup after a crash. In particular, scanning the redo log and applying the redo log are faster than in MySQL 5.1 and earlier, due to improved algorithms for memory management. You do not need to take any actions to take advantage of this performance enhancement. If you kept the size of your redo log files artificially low because recovery took a long time, you can consider increasing the file size.
For more information about InnoDB recovery, see
Section 13.3.7.1, “The InnoDB
Recovery Process”.
Starting with InnoDB 1.1 with MySQL 5.5, you can profile certain internal InnoDB operations using the MySQL Performance Schema feature. This type of tuning is primarily for expert users, those who push the limits of MySQL performance, read the MySQL source code, and evaluate optimization strategies to overcome performance bottlenecks. DBAs can also use this feature for capacity planning, to see whether their typical workload encounters any performance bottlenecks with a particular combination of CPU, RAM, and disk storage; and if so, to judge whether performance can be improved by increasing the capacity of some part of the system.
To use this feature to examine InnoDB performance:
You must be running MySQL 5.5 or higher. You must build the database server from source, enabling the Performance Schema feature by building with the
--with-perfschema
option. Since the Performance Schema feature introduces some performance overhead, you should use it on a test or development system rather than on a production system.You must be running InnoDB 1.1 or higher.
You must be generally familiar with how to use the Performance Schema feature, for example to query tables in the
performance_schema
database.Examine the following kinds of InnoDB objects by querying the appropriate
performance_schema
tables. The items associated with InnoDB all contain the substringinnodb
in theNAME
column.For the definitions of the
*_instances
tables, see Section 20.7.2, “Performance Schema Instance Tables”. For the definitions of the*_summary_*
tables, see Section 20.7.4, “Performance Schema Summary Tables”. For the definition of thethread
table, see Section 20.7.5, “Performance Schema Miscellaneous Tables”. For the definition of the*_current_*
and*_history_*
tables, see Section 20.7.3, “Performance Schema Wait Event Tables”.Mutexes in the
mutex_instances
table. (Mutexes and RW-locks related to theInnoDB
buffer pool are not included in this coverage; the same applies to the output of theSHOW ENGINE INNODB MUTEX
command.)RW-locks in the
rwlock_instances
table.RW-locks in the
rwlock_instances
table.File I/O operations in the
file_instances
,file_summary_by_event_name
, andfile_summary_by_instance
tables.Threads in the
PROCESSLIST
table.
During performance testing, examine the performance data in the
events_waits_current
andevents_waits_history_long
tables. If you are interested especially in InnoDB-related objects, use the clausewhere name like "%innodb%"
to see just those entries; otherwise, examine the performance statistics for the overall MySQL server.You must be running MySQL 5.5, with the Performance Schema enabled by building with the
--with-perfschema
build option.
For more information about the MySQL Performance Schema, see Глава 20, MySQL Performance Schema.
This performance enhancement is primarily useful for people with a
large buffer pool size,
typically in the multi-gigabyte range. To take advantage of this
speedup, you must set the new
innodb_buffer_pool_instances
configuration option, and you might also adjust the
innodb_buffer_pool_size
value.
When the InnoDB buffer pool is large, many data requests can be satisfied by retrieving from memory. You might encounter bottlenecks from multiple threads trying to access the buffer pool at once. Starting in InnoDB 1.1 and MySQL 5.5, you can enable multiple buffer pools to minimize this contention. Each page that is stored in or read from the buffer pool is assigned to one of the buffer pools randomly, using a hashing function. Each buffer pool manages its own free lists, flush lists, LRUs, and all other data structures connected to a buffer pool, and is protected by its own buffer pool mutex.
To enable this feature, set the
innodb_buffer_pool_instances
configuration
option to a value greater than 1 (the default) up to 64 (the
maximum). This option only takes effect when you set the
innodb_buffer_pool_size
to a size of 1 gigabyte
or more. The total size you specify is divided up among all the
buffer pools. We recommend specifying a combination of
innodb_buffer_pool_instances
and
innodb_buffer_pool_size
so that each buffer
pool instance is at least 1 gigabyte.
For more information about the InnoDB buffer pool, see
Section 7.9.1, “The InnoDB
Buffer Pool”.
Starting in InnoDB 1.1 with MySQL 5.5, the limit on concurrent transactions is greatly expanded, removing a bottleneck with the InnoDB rollback segment that affected high-capacity systems. The limit applies to concurrent transactions that change any data; read-only transactions do not count against that maximum.
The single rollback segment is now divided into 128 segments, each of which can support up to 1023 transactions that perform writes, for a total of approximately 128K concurrent transactions. The original transaction limit was 1023.
Each transaction is assigned to one of the rollback segments, and remains tied to that rollback segment for the duration. This enhancement improves both scalability (higher number of concurrent transactions) and performance (less contention when different transactions access the rollback segments).
To take advantage of this feature, you do not need to create any new database or tables, or reconfigure anything. You must do a slow shutdown before upgrading from MySQL 5.1 or earlier, or some time afterward. InnoDB makes the required changes inside the system tablespace automatically, the first time you restart after performing a slow shutdown.
For more information about performance of InnoDB under high
transactional load, see
Section 7.5.2, “Optimizing InnoDB
Transaction Management”.
Starting in InnoDB 1.1 with MySQL 5.5, the purge operations (a type of garbage collection) that InnoDB performs automatically can be done in a separate thread, rather than as part of the master thread. This change improves scalability, because the main database operations run independently from maintenance work happening in the background.
To enable this feature, set the configuration option
innodb_purge_threads=1
, as opposed to the
default of 0, which combines the purge operation into the master
thread.
You might not notice a significant speedup, because the purge
thread might encounter new types of contention; the single purge
thread really lays the groundwork for further tuning and possibly
multiple purge threads in the future. There is another new
configuration option, innodb_purge_batch_size
with a default of 20 and maximum of 5000. This option is mainly
intended for experimentation and tuning of purge operations, and
should not be interesting to typical users.
For more information about InnoDB I/O performance, see
Section 7.5.7, “Optimizing InnoDB
Disk I/O”.
This is another performance improvement that comes for free, with no user action or configuration needed. The details here are intended for performance experts who delve into the InnoDB source code, or interpret reports with keywords such as “mutex” and “log_sys”.
The mutex known as the log sys
mutex has historically done double duty, controlling access to
internal data structures related to log records and the
LSN, as well as pages in the
buffer pool that are
changed when a
mini-transaction is
committed. Starting in InnoDB 1.1 with MySQL 5.5, these two kinds
of operations are protected by separate mutexes, with a new
log_buf
mutex controlling writes to buffer pool
pages due to mini-transactions.
For performance considerations for InnoDB locking operations, see Section 7.10, “Optimizing Locking Operations”.
Starting with InnoDB 1.1 with MySQL 5.5, concurrent access to the buffer pool is faster. Operations involving the flush list, a data structure related to the buffer pool, are now controlled by a separate mutex and do not block access to the buffer pool. You do not need to configure anything to take advantage of this speedup; it is fully automatic.
For more information about the InnoDB buffer pool, see
Section 7.9.1, “The InnoDB
Buffer Pool”.
- 13.4.8.1. The Barracuda File Format
- 13.4.8.2. Dynamic Control of System Configuration Parameters
- 13.4.8.3.
TRUNCATE TABLE
Reclaims Space - 13.4.8.4. InnoDB Strict Mode
- 13.4.8.5. Controlling Optimizer Statistics Estimation
- 13.4.8.6. Better Error Handling when Dropping Indexes
- 13.4.8.7. More Compact Output of
SHOW ENGINE INNODB MUTEX
- 13.4.8.8. More Read-Ahead Statistics
This chapter describes several recently added InnoDB features that
offer new flexibility and improve ease of use, reliability and
performance. The Barracuda
file format improves efficiency for storing large variable-length
columns, and enables table compression. Configuration options that
once were unchangeable after startup, are now flexible and can be
changed dynamically. Some improvements are automatic, such as faster
and more efficient TRUNCATE TABLE
. Others allow
you the flexibility to control InnoDB behavior; for example, you can
control whether certain problems cause errors or just warnings. And
informational messages and error reporting continue to be made more
user-friendly.
InnoDB has started using named file formats to improve
compatibility in upgrade and downgrade situations, or
heterogeneous systems running different levels of MySQL. Many
important InnoDB features, such as table compression and the
DYNAMIC
row format for more efficient BLOB
storage, require creating tables in the
Barracuda file format.
The original file format, which previously didn't have a name, is
known now as Antelope.
To create new tables that take advantage of the Barracuda
features, enable that file format using the configuration
parameter innodb_file_format
. The
value of this parameter determines whether a newly created table
or index can use compression or the new DYNAMIC
row format.
To preclude the use of new features that would make your database
inaccessible to the built-in InnoDB in MySQL 5.1 and prior
releases, omit innodb_file_format
or set it to Antelope.
You can set the value of
innodb_file_format
on the
command line when you start mysqld
, or in the
option file my.cnf
(Unix operating systems) or
my.ini
(Windows). You can also change it
dynamically with the SET GLOBAL
statement.
For more information about managing file formats, see Section 13.4.4, “InnoDB File-Format Management”.
In MySQL 5.5 and higher, you can change certain system configuration parameters without shutting down and restarting the server, as was necessary in MySQL 5.1 and lower. This increases uptime, and makes it easier to test and prototype new SQL and application code. The following sections explain these parameters.
Since MySQL version 4.1, InnoDB has provided two alternatives
for how tables are stored on disk. You can create a new table
and its indexes in the shared
system
tablespace, physically stored in the
ibdata files. Or, you
can store a new table and its indexes in a separate tablespace
(a .ibd file). The storage
layout for each InnoDB table is determined by the configuration
parameter
innodb_file_per_table
at
the time the table is created.
In MySQL 5.5 and higher, the configuration parameter
innodb_file_per_table
is
dynamic, and can be set ON
or
OFF
using the SET GLOBAL
.
Previously, the only way to set this parameter was in the MySQL
option file (my.cnf
or
my.ini
), and changing it required shutting
down and restarting the server.
The default setting is OFF
, so new tables and
indexes are created in the system tablespace. Dynamically
changing the value of this parameter requires the
SUPER
privilege and immediately affects the
operation of all connections.
Tables created when
innodb_file_per_table
is
enabled can use the
Barracuda file format, and
TRUNCATE
returns the disk space for those
tables to the operating system. The Barracuda file format in
turn enables features such as table compression and the
DYNAMIC
row format. Tables created when
innodb_file_per_table
is off
cannot use these features. To take advantage of those features
for an existing table, you can turn on the file-per-table
setting and run ALTER TABLE
for that
table.
t
ENGINE=INNODB
When you redefine the primary key for an InnoDB table, the table
is re-created using the current settings for
innodb_file_per_table
and
innodb_file_format
. This
behavior does not apply when adding or dropping InnoDB secondary
indexes, as explained in Section 13.4.2, “Fast Index Creation in the InnoDB Storage Engine”.
When a secondary index is created without rebuilding the table,
the index is stored in the same file as the table data,
regardless of the current
innodb_file_per_table
setting.
In MySQL 5.5 and higher, you can change the setting of
innodb_stats_on_metadata
dynamically at runtime, to control whether or not InnoDB
performs statistics gathering when metadata statements are
executed. To change the setting, issue the statement
SET GLOBAL
innodb_stats_on_metadata=
,
where mode
is
either mode
ON
or OFF
(or
1
or 0
). Changing this
setting requires the SUPER
privilege and
immediately affects the operation of all connections.
This setting is related to the feature described in Section 13.4.8.5, “Controlling Optimizer Statistics Estimation”.
The length of time a transaction waits for a resource, before
giving up and rolling back the statement, is determined by the
value of the configuration parameter
innodb_lock_wait_timeout
.
(In MySQL 5.0.12 and earlier, the entire transaction was rolled
back, not just the statement.) Your application can try the
statement again (usually after waiting for a while), or roll
back the entire transaction and restart.
The error returned when the timeout period is exceeded is:
ERROR HY000: Lock wait timeout exceeded; try restarting transaction
In MySQL 5.5 and higher, the configuration parameter
innodb_lock_wait_timeout
can be
set at runtime with the SET GLOBAL
or
SET SESSION
statement. Changing the
GLOBAL
setting requires the
SUPER
privilege and affects the operation of
all clients that subsequently connect. Any client can change the
SESSION
setting for
innodb_lock_wait_timeout
, which
affects only that client.
In MySQL 5.1 and earlier, the only way to set this parameter was
in the MySQL option file (my.cnf
or
my.ini
), and changing it required shutting
down and restarting the server.
As described in Section 13.4.7.5, “Controlling Adaptive Hash Indexing”, it may be desirable, depending on your workload, to dynamically enable or disable the adaptive hash indexing scheme InnoDB uses to improve query performance.
The start-up option
innodb_adaptive_hash_index
allows the adaptive hash index to be disabled. It is enabled by
default. You can modify this parameter through the SET
GLOBAL
statement, without restarting the server.
Changing the setting requires the SUPER
privilege.
Disabling the adaptive hash index empties the hash table immediately. Normal operations can continue while the hash table is emptied, and executing queries that were using the hash table access the index B-trees directly instead. When the adaptive hash index is re-enabled, the hash table is populated again during normal operation.
When you truncate a table
that is stored in a
.ibd
file
of its own (because
innodb_file_per_table
was enabled
when the table was created), and if the table is not referenced in
a FOREIGN KEY
constraint, the table is dropped
and re-created in a new .ibd
file. This
operation is much faster than deleting the rows one by one. The
operating system can reuse the disk space, in contrast to tables
within the InnoDB system
tablespace, where only InnoDB can use the space after they
are truncated. Physical
backups can also be smaller, without big blocks of unused
space in the middle of the system tablespace.
Previous versions of InnoDB would re-use the existing
.ibd
file, thus releasing the space only to
InnoDB for storage management, but not to the operating system.
Note that when the table is truncated, the count of rows affected
by the TRUNCATE TABLE
statement is an arbitrary
number.
If there is a referential constraint between two columns in the same table, that table can still be truncated using this fast technique.
If there are referential constraints between the table being
truncated and other tables, the truncate operation fails. This
is a change to the previous behavior, which would transform the
TRUNCATE
operation to a
DELETE
operation that removed all the rows
and triggered ON DELETE
operations on
child tables.
To guard against ignored typos and syntax errors in SQL, or other
unintended consequences of various combinations of operational
modes and SQL statements, InnoDB provides a
strict mode of
operations. In this mode, InnoDB raises error conditions in
certain cases, rather than issuing a warning and processing the
specified statement (perhaps with unintended behavior). This is
analogous to sql_mode
in MySQL, which controls
what SQL syntax MySQL accepts, and determines whether it silently
ignores errors, or validates input syntax and data values. Since
strict mode is relatively new, some statements that execute
without errors with earlier versions of MySQL might generate
errors unless you disable strict mode.
The setting of InnoDB strict mode affects the handling of syntax
errors on the CREATE TABLE
,
ALTER TABLE
and
CREATE INDEX
statements. The
strict mode also enables a record size check, so that an
INSERT
or UPDATE
never fails
due to the record being too large for the selected page size.
We recommend running in strict mode when using the
ROW_FORMAT
and
KEY_BLOCK_SIZE
clauses on
CREATE TABLE
,
ALTER TABLE
, and
CREATE INDEX
statements. Without
strict mode, InnoDB ignores conflicting clauses and creates the
table or index, with only a warning in the message log. The
resulting table might have different behavior than you intended,
such as having no compression when you tried to create a
compressed table. When InnoDB strict mode is on, such problems
generate an immediate error and the table or index is not created,
avoiding a troubleshooting session later.
InnoDB strict mode is set with the configuration parameter
innodb_strict_mode
, which can be
specified as on
or off
. You
can set the value on the command line when you start
mysqld
, or in the configuration file
my.cnf
or my.ini
. You can
also enable or disable InnoDB strict mode at run time with the
statement SET [GLOBAL|SESSION]
innodb_strict_mode=
,
where mode
is either
mode
ON
or OFF
. Changing the
GLOBAL
setting requires the
SUPER
privilege and affects the operation of
all clients that subsequently connect. Any client can change the
SESSION
setting for
innodb_strict_mode
, and the
setting affects only that client.
The MySQL query optimizer uses estimated statistics about key
distributions to choose the indexes for an execution plan, based
on the relative
selectivity of the index.
Certain operations cause InnoDB to sample random pages from each
index on a table to estimate the
cardinality of the index.
(This technique is known as
random dives.) These
operations include the ANALYZE
TABLE
statement, the SHOW TABLE
STATUS
statement, and accessing the table for the first
time after a restart.
To give you control over the quality of the statistics estimate
(and thus better information for the query optimizer), you can now
change the number of sampled pages using the parameter
innodb_stats_sample_pages
.
Previously, the number of sampled pages was always 8, which could
be insufficient to produce an accurate estimate, leading to poor
index choices by the query optimizer. This technique is especially
important for large tables and tables used in
joins. Unnecessary
full table scans for
such tables can be a substantial performance issue.
You can set the global parameter
innodb_stats_sample_pages
, at
run time. The default value for this parameter is 8, preserving
the same behavior as in past releases.
The value of
innodb_stats_sample_pages
affects the index sampling for all tables
and indexes. There are the following potentially significant
impacts when you change the index sample size:
Small values like 1 or 2 can result in very inaccurate estimates of cardinality.
Increasing the
innodb_stats_sample_pages
value might require more disk reads. Values much larger than 8 (say, 100), can cause a big slowdown in the time it takes to open a table or executeSHOW TABLE STATUS
.The optimizer might choose very different query plans based on different estimates of index selectivity.
To disable the cardinality estimation for metadata statements such
as SHOW TABLE STATUS
, execute the statement
SET GLOBAL innodb_stats_on_metadata=OFF
(or
0
). The ability to set this option dynamically
is also relatively new.
All InnoDB tables are opened, and the statistics are re-estimated
for all associated indexes, when the mysql
client starts if the auto-rehash setting is set on (the default).
To improve the start up time of the mysql
client, you can turn auto-rehash off. The auto-rehash feature
enables automatic name completion of database, table, and column
names for interactive users.
Whatever value of
innodb_stats_sample_pages
works
best for a system, set the option and leave it at that value.
Choose a value that results in reasonably accurate estimates for
all tables in your database without requiring excessive I/O.
Because the statistics are automatically recalculated at various
times other than on execution of ANALYZE
TABLE
, it does not make sense to increase the index
sample size, run ANALYZE TABLE
,
then decrease sample size again. The more accurate statistics
calculated by ANALYZE
running with a high value
of innodb_stats_sample_pages
can
be wiped away later.
Although it is not possible to specify the sample size on a
per-table basis, smaller tables generally require fewer index
samples than larger tables do. If your database has many large
tables, consider using a higher value for
innodb_stats_sample_pages
than if
you have mostly smaller tables.
For optimal performance with DML statements, InnoDB requires an
index to exist on foreign
key columns, so that UPDATE
and
DELETE
operations on a
parent table can easily
check whether corresponding rows exist in the
child table. MySQL creates
or drops such indexes automatically when needed, as a side-effect
of CREATE TABLE
,
CREATE INDEX
, and
ALTER TABLE
statements.
When you drop an index, InnoDB checks whether the index is not used for checking a foreign key constraint. It is still OK to drop the index if there is another index that can be used to enforce the same constraint. InnoDB prevents you from dropping the last index that can enforce a particular referential constraint.
The message that reports this error condition is:
ERROR 1553 (HY000): Cannot drop index 'fooIdx
':
needed in a foreign key constraint
This message is friendlier than the earlier message it replaces:
ERROR 1025 (HY000): Error on rename of './db2/#sql-18eb_3
' to './db2/foo
'(errno: 150)
A similar change in error reporting applies to an attempt to drop
the primary key index. For tables without an explicit
PRIMARY KEY
, InnoDB creates an implicit
clustered index using
the first columns of the table that are declared
UNIQUE
and NOT NULL
. When
you drop such an index, InnoDB automatically copies the table and
rebuilds the index using a different UNIQUE NOT
NULL
group of columns or a system-generated key. Since
this operation changes the primary key, it uses the slow method of
copying the table and re-creating the index, rather than the Fast
Index Creation technique from
Section 13.4.2.3, “Implementation Details of Fast Index Creation”.
Previously, an attempt to drop an implicit clustered index (the
first UNIQUE NOT NULL
index) failed if the
table did not contain a PRIMARY KEY
:
ERROR 42000: This table type requires a primary key
The statement SHOW ENGINE INNODB MUTEX
displays
information about InnoDB mutexes
and rw-locks. Although this
information is useful for tuning on multi-core systems, the amount
of output can be overwhelming on systems with a big
buffer pool. There is one
mutex and one rw-lock in each 16K buffer pool block, and there are
65,536 blocks per gigabyte. It is unlikely that a single block
mutex or rw-lock from the buffer pool could become a performance
bottleneck.
SHOW ENGINE INNODB MUTEX
now skips the mutexes
and rw-locks of buffer pool blocks. It also does not list any
mutexes or rw-locks that have never been waited on
(os_waits=0
). Thus, SHOW ENGINE INNODB
MUTEX
only displays information about mutexes and
rw-locks outside of the buffer pool that have caused at least one
OS-level wait.
As described in Section 13.4.7.7, “Changes in the Read-Ahead Algorithm”, a
read-ahead request is an asynchronous I/O request issued in
anticipation that a page will be used in the near future. Knowing
how many pages are read through this read-ahead mechanism, and how
many of them are evicted from the buffer pool without ever being
accessed, can be useful to help fine-tune the parameter
innodb_read_ahead_threshold
.
SHOW ENGINE INNODB STATUS
output displays the
global status variables
Innodb_buffer_pool_read_ahead
and
Innodb_buffer_pool_read_ahead_evicted
. These
variables indicate the number of pages brought into the
buffer pool by read-ahead
requests, and the number of such pages
evicted from the buffer pool
without ever being accessed respectively. These counters provide
global values since the last server restart.
SHOW ENGINE INNODB INNODB STATUS
also shows the
rate at which the read-ahead pages are read in and the rate at
which such pages are evicted without being accessed. The
per-second averages are based on the statistics collected since
the last invocation of SHOW ENGINE INNODB INNODB
STATUS
and are displayed in the BUFFER POOL AND
MEMORY
section of the output.
Since the InnoDB read-ahead mechanism has been simplified to
remove random read-ahead, the status variables
Innodb_buffer_pool_read_ahead_rnd
and
Innodb_buffer_pool_read_ahead_seq
are no longer
part of the SHOW ENGINE INNODB STATUS
output.
When you use the InnoDB storage engine 1.1 and above, with MySQL 5.5 and above, you do not need to do anything special to install: everything comes configured as part of the MySQL source and binary distributions. This is a change from earlier releases of the InnoDB Plugin, where you were required to match up MySQL and InnoDB version numbers and update your build and configuration processes.
The InnoDB storage engine is included in the MySQL distribution, starting from MySQL 5.1.38. From MySQL 5.1.46 and up, this is the only download location for the InnoDB storage engine; it is not available from the InnoDB web site.
If you used any scripts or configuration files with the earlier
InnoDB storage engine from the InnoDB web site, be aware that the filename
of the shared library as supplied by MySQL is
ha_innodb_plugin.so
or
ha_innodb_plugin.dll
, as opposed to
ha_innodb.so
or ha_innodb.dll
in the older Plugin downloaded from the InnoDB web site. You might
need to change the applicable file names in your startup or
configuration scripts.
Because the InnoDB storage engine has now replaced the built-in InnoDB,
you no longer need to specify options like
--ignore-builtin-innodb
and
--plugin-load
during startup.
To take best advantage of current InnoDB features, we recommend specifying the following options in your configuration file:
innodb_file_per_table=1 innodb_file_format=barracuda innodb_strict_mode=1
For information about these new features, see
Section 13.4.8.2.1, “Dynamically Changing innodb_file_per_table
”,
Section 13.4.4, “InnoDB File-Format Management”, and
Section 13.4.8.4, “InnoDB Strict Mode”. You might need
to continue to use the previous values for these parameters in some
replication and similar configurations involving both new and older
versions of MySQL.
Prior to MySQL 5.5, some upgrade scenarios involved upgrading the separate instance of InnoDB known as the InnoDB Plugin. In MySQL 5.5 and higher, the features of the InnoDB Plugin have been folded back into built-in InnoDB, so the upgrade procedure for InnoDB is the same as the one for the MySQL server. For details, see Section 2.11.1, “Upgrading MySQL”.
Prior to MySQL 5.5, some downgrade scenarios involved switching the separate instance of InnoDB known as the InnoDB Plugin back to the built-in InnoDB storage engine. In MySQL 5.5 and higher, the features of the InnoDB Plugin have been folded back into built-in InnoDB, so the downgrade procedure for InnoDB is the same as the one for the MySQL server. For details, see Section 2.11.2, “Downgrading MySQL”.
Since InnoDB 1.1 is tightly integrated with MySQL 5.5, for changes after the initial InnoDB 1.1 release, see the MySQL 5.5 Reference Manual, Section D.1, “Changes in Release 5.5.x (Production)”.
For an overview of the changes, see this introduction article for MySQL 5.5 with InnoDB 1.1. The following is a condensed version of the change log.
Fix for bug #52580: Crash in ha_innobase::open on executing INSERT with concurrent ALTER TABLE.
Change in MySQL bug #51557 releases the mutex LOCK_open before ha_innobase::open(), causing racing condition for index translation table creation. Fix it by adding dict_sys mutex for the operation.
Add support for multiple buffer pools.
Fix Bug #26590: MySQL does not allow more than 1023 open transactions. Create additional rollback segments on startup. Reduce the upper limit of total rollback segments from 256 to 128. This is because we can't use the sign bit. It has not caused problems in the past because we only created one segment. InnoDB has always had the capability to use the additional rollback segments, therefore this patch is backward compatible. The only requirement to maintain backward compatibility has been to ensure that the additional segments are created after the double write buffer. This is to avoid breaking assumptions in the existing code.
Implement Performance Schema in InnoDB. Objects in four different modules in InnoDB have been performance instrumented, these modules are: mutexes, rwlocks, file I/O, and threads We mostly preserved the existing APIs, but APIs would point to instrumented function wrappers if performance schema is defined. There are 4 different defines that controls the instrumentation of each module. The feature is off by default, and will be compiled in with special build option, and requre configure option to turn it on when server boots.
Implement the buf_pool_watch for DeleteBuffering in the page hash table. This serves two purposes. It allows multiple watches to be set at the same time (by multiple purge threads) and it removes a race condition when the read of a block completes about the time the buffer pool watch is being set.
Introduce a new mutex to protect flush_list. Redesign mtr_commit() in a way that log_sys mutex is not held while all mtr_memos are popped and is released just after the modified blocks are inserted into the flush_list. This should reduce contention on log_sys mutex.
Implement the global variable
innodb_change_buffering
, with the
following values:
none
: buffer nothinginserts
: buffer inserts (like InnoDB so far)deletes
: buffer delete-markschanges
: buffer inserts and delete-markspurges
: buffer delete-marks and deletesall
: buffer all operations (insert, delete-mark, delete)
The default is all
. All values except
none
and inserts
will make
InnoDB write new-format records to the insert buffer, even for
inserts.
Provide support for native AIO on Linux.
The InnoDB 1.0.x releases that accompany MySQL 5.1 have their own change history. Changes up to InnoDB 1.0.8 are listed at http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes.html. Changes from InnoDB 1.0.9 and up are listed in Changes in Release 5.1.x (Production), incorporated into the main MySQL change log.
Oracle acknowledges that certain Third Party and Open Source software has been used to develop or is incorporated in the InnoDB storage engine. This appendix includes required third-party license information.
Oracle gratefully acknowledges the following contributions from Google, Inc. to improve InnoDB performance:
Replacing InnoDB's use of Pthreads mutexes with calls to GCC atomic builtins, as discussed in Section 13.4.7.2, “Faster Locking for Improved Scalability”. This change means that InnoDB mutex and and rw-lock operations take less CPU time, and improves throughput on those platforms where the atomic operations are available.
Controlling master thread
I/O
rate, as discussed in Section 13.4.7.11, “Controlling the Master Thread I/O Rate”. The master thread in InnoDB is a thread that performs various tasks in the background. Historically, InnoDB has used a hard coded value as the totalI/O
capacity of the server. With this change, user can control the number ofI/O
operations that can be performed per second based on their own workload.
Changes from the Google contributions were incorporated in the
following source code files: btr0cur.c
,
btr0sea.c
, buf0buf.c
,
buf0buf.ic
, ha_innodb.cc
,
log0log.c
, log0log.h
,
os0sync.h
, row0sel.c
,
srv0srv.c
, srv0srv.h
,
srv0start.c
, sync0arr.c
,
sync0rw.c
, sync0rw.h
,
sync0rw.ic
, sync0sync.c
,
sync0sync.h
, sync0sync.ic
,
and univ.i
.
These contributions are incorporated subject to the conditions
contained in the file COPYING.Google
, which are
reproduced here.
Copyright (c) 2008, 2009, Google Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials
provided with the distribution.
* Neither the name of the Google Inc. nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Oracle gratefully acknowledges the contribution of Percona, Inc. to improve InnoDB performance by implementing configurable background threads, as discussed in Section 13.4.7.8, “Multiple Background I/O Threads”. InnoDB uses background threads to service various types of I/O requests. The change provides another way to make InnoDB more scalable on high end systems.
Changes from the Percona, Inc. contribution were incorporated in
the following source code files: ha_innodb.cc
,
os0file.c
, os0file.h
,
srv0srv.c
, srv0srv.h
, and
srv0start.c
.
This contribution is incorporated subject to the conditions
contained in the file COPYING.Percona
, which
are reproduced here.
Copyright (c) 2008, 2009, Percona Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials
provided with the distribution.
* Neither the name of the Percona Inc. nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Oracle gratefully acknowledges the following contributions from Sun Microsystems, Inc. to improve InnoDB performance:
Introducing the PAUSE instruction inside spin loops, as discussed in Section 13.4.7.13, “Using the PAUSE Instruction in InnoDB Spin Loops”. This change increases performance in high concurrency, CPU-bound workloads.
Enabling inlining of functions and prefetch with Sun Studio.
Changes from the Sun Microsystems, Inc. contribution were
incorporated in the following source code files:
univ.i
, ut0ut.c
, and
ut0ut.h
.
This contribution is incorporated subject to the conditions
contained in the file COPYING.Sun_Microsystems
,
which are reproduced here.
Copyright (c) 2009, Sun Microsystems, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials
provided with the distribution.
* Neither the name of Sun Microsystems, Inc. nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Throughout the course of development, InnoDB 1.1 and its predecessor the InnoDB Plugin introduced new configuration parameters. The following table summarizes those parameters:
Table 13.8. InnoDB 1.1 New Parameter Summary
Name | Cmd-Line | Option File | System Var | Scope | Dynamic | Default |
---|---|---|---|---|---|---|
innodb_adaptive_flushing | YES | YES | YES | GLOBAL | YES | TRUE |
innodb_buffer_pool_instances | YES | YES | YES | GLOBAL | YES | TRUE |
innodb_change_buffering | YES | YES | YES | GLOBAL | YES | inserts |
innodb_file_format | YES | YES | YES | GLOBAL | YES | Antelope |
innodb_file_format_check | YES | YES | YES | GLOBAL | NO | 1 |
innodb_file_format_max | YES | YES | YES | GLOBAL | YES | Antelope for a new database;
Barracuda if any tables using that file
format exist in the database |
innodb_io_capacity | YES | YES | YES | GLOBAL | YES | 200 |
innodb_old_blocks_pct | YES | YES | YES | GLOBAL | YES | 37 |
innodb_old_blocks_time | YES | YES | YES | GLOBAL | YES | 0 |
innodb_purge_batch_size | YES | YES | YES | GLOBAL | YES | 0 |
innodb_purge_threads | YES | YES | YES | GLOBAL | YES | 0 |
innodb_read_ahead_threshold | YES | YES | YES | GLOBAL | YES | 56 |
innodb_read_io_threads | YES | YES | YES | GLOBAL | NO | 4 |
innodb_spin_wait_delay | YES | YES | YES | GLOBAL | YES | 6 |
innodb_stats_sample_pages | YES | YES | YES | GLOBAL | YES | 8 |
innodb_strict_mode | YES | YES | YES | GLOBAL|SESSION | YES | FALSE |
innodb_use_native_aio | YES | YES | YES | GLOBAL | NO | TRUE |
innodb_use_sys_malloc | YES | YES | YES | GLOBAL | NO | TRUE |
innodb_write_io_threads | YES | YES | YES | GLOBAL | NO | 4 |
Whether InnoDB uses a new algorithm to estimate the required rate of flushing. The default value is
TRUE
. This parameter was added in InnoDB storage engine 1.0.4. See Section 13.4.7.12, “Controlling the Flushing Rate of Dirty Pages” for more information.Whether InnoDB performs insert buffering. The default value is
"inserts"
(buffer insert operations). This parameter was added in InnoDB storage engine 1.0.3. See Section 13.4.7.4, “Controlling InnoDB Change Buffering” for more information.The default file format for new InnoDB tables. The default is Antelope. To enable support for table compression, change it to Barracuda. This parameter was added in InnoDB storage engine 1.0.1. See Section 13.4.4.1, “Enabling File Formats” for more information.
innodb_file_format_check
andinnodb_file_format_max
Controls whether InnoDB performs file format compatibility checking when opening a database. The default value is
innodb-file-format-check=1
, withinnodb_file_format_max
set to the highest format that is used in the database (either Barracuda or Antelope). See Section 13.4.4.2.1, “Compatibility Check When InnoDB Is Started” for more information.The number of
I/O
operations that can be performed per second. The allowable value range is any number 100 or greater, and the default value is200
. This parameter was added in InnoDB storage engine 1.0.4. To reproduce the earlier behavior, use a value of 100. See Section 13.4.7.11, “Controlling the Master Thread I/O Rate” for more information.Controls the desired percentage of “old” blocks in the LRU list of the buffer pool. The default value is
37
and the allowable value range is5 to 95
. This parameter was added in InnoDB storage engine 1.0.5. See Section 13.4.7.15, “Making Buffer Pool Scan Resistant” for more information.The time in milliseconds since the first access to a block during which it can be accessed again without being made “young”. The default value is
0
which means that blocks are moved to the “young” end of the LRU list at the first access. This parameter was added in InnoDB storage engine 1.0.5. See Section 13.4.7.15, “Making Buffer Pool Scan Resistant” for more information.Control the sensitivity of the linear read ahead. The allowable value range is
0
to64
and the default value is56
. This parameter was added in InnoDB storage engine 1.0.4. See Section 13.4.7.7, “Changes in the Read-Ahead Algorithm” for more information.The number of background
I/O
threads used for reads. The allowable value range is1
to64
and the default value is4
. This parameter was added in InnoDB storage engine 1.0.4. See Section 13.4.7.8, “Multiple Background I/O Threads” for more information.Maximum delay between polling for a spin lock. The allowable value range is
0
(meaning unlimited) or positive integers and the default value is6
. This parameter was added in InnoDB storage engine 1.0.4. See Section 13.4.7.14, “Control of Spin Lock Polling” for more information.The number of index pages to sample when calculating statistics. The allowable value range is
1-unlimited
and the default value is8
. This parameter was added in InnoDB storage engine 1.0.2. See Section 13.4.8.5, “Controlling Optimizer Statistics Estimation” for more information.Whether InnoDB raises error conditions in certain cases, rather than issuing a warning. This parameter was added in InnoDB storage engine 1.0.2. See Section 13.4.8.4, “InnoDB Strict Mode” for more information.
Whether InnoDB uses its own memory allocator or an allocator of the operating system. The default value is
ON
(use an allocator of the underlying system). This parameter was added in InnoDB storage engine 1.0.3. See Section 13.4.7.3, “Using Operating System Memory Allocators” for more information.The number of background
I/O
threads used for writes. The allowable value range is1
to64
and the default value is4
. This parameter was added in InnoDB storage engine 1.0.4. See Section 13.4.7.8, “Multiple Background I/O Threads” for more information.
Beginning in InnoDB storage engine 1.0.4, the following configuration parameter has been removed:
This parameter has been replaced by two new parameters
innodb_read_io_threads
andinnodb_write_io_threads
. See Section 13.4.7.8, “Multiple Background I/O Threads” for more information.
For better out-of-the-box performance, the following InnoDB configuration parameters have new default values since MySQL 5.1:
Table 13.9. InnoDB Parameters with New Defaults
Name | Old Default | New Default |
---|---|---|
innodb_additional_mem_pool_size | 1MB | 8MB |
innodb_buffer_pool_size | 8MB | 128MB |
innodb_change_buffering | inserts | all |
innodb_file_format_check | ON | 1 |
innodb_log_buffer_size | 1MB | 8MB |
innodb_max_dirty_pages_pct | 90 | 75 |
innodb_sync_spin_loops | 20 | 30 |
innodb_thread_concurrency | 8 | 0 |
Before MySQL 5.5.5, MyISAM
is the default storage
engine. (The default was changed to InnoDB
in
MySQL 5.5.5.) MyISAM
is based on the older (and
no longer available) ISAM
storage engine but has
many useful extensions.
Table 13.10. MyISAM
Storage Engine
Features
Storage limits | 256TB | Transactions | No | Locking granularity | Table |
MVCC | No | Geospatial data type support | Yes | Geospatial indexing support | Yes |
B-tree indexes | Yes | Hash indexes | No | Full-text search indexes | Yes |
Clustered indexes | No | Data caches | No | Index caches | Yes |
Compressed data | Yes[a] | Encrypted data[b] | Yes | Cluster database support | No |
Replication support[c] | Yes | Foreign key support | No | Backup / point-in-time recovery[d] | Yes |
Query cache support | Yes | Update statistics for data dictionary | Yes | ||
[a] Compressed MyISAM tables are supported only when using the compressed row format. Tables using the compressed row format with MyISAM are read only. [b] Implemented in the server (via encryption functions), rather than in the storage engine. [c] Implemented in the server, rather than in the storage product. [d] Implemented in the server, rather than in the storage product. |
Each MyISAM
table is stored on disk in three
files. The files have names that begin with the table name and have
an extension to indicate the file type. An .frm
file stores the table format. The data file has an
.MYD
(MYData
) extension. The
index file has an .MYI
(MYIndex
) extension.
To specify explicitly that you want a MyISAM
table, indicate that with an ENGINE
table option:
CREATE TABLE t (i INT) ENGINE = MYISAM;
As of MySQL 5.5.5, it is normally necessary to use
ENGINE
to specify the MyISAM
storage engine because InnoDB
is the default
engine. Before 5.5.5, this is unnecessary because
MyISAM
is the default engine unless the default
has been changed. To ensure that MyISAM
is used
in situations where the default might have been changed, include the
ENGINE
option explicitly.
You can check or repair MyISAM
tables with the
mysqlcheck client or myisamchk
utility. You can also compress MyISAM
tables with
myisampack to take up much less space. See
Section 4.5.3, “mysqlcheck — A Table Maintenance Program”, Section 4.6.3, “myisamchk — MyISAM Table-Maintenance Utility”, and
Section 4.6.5, “myisampack — Generate Compressed, Read-Only MyISAM Tables”.
MyISAM
tables have the following characteristics:
All data values are stored with the low byte first. This makes the data machine and operating system independent. The only requirements for binary portability are that the machine uses two's-complement signed integers and IEEE floating-point format. These requirements are widely used among mainstream machines. Binary compatibility might not be applicable to embedded systems, which sometimes have peculiar processors.
There is no significant speed penalty for storing data low byte first; the bytes in a table row normally are unaligned and it takes little more processing to read an unaligned byte in order than in reverse order. Also, the code in the server that fetches column values is not time critical compared to other code.
All numeric key values are stored with the high byte first to permit better index compression.
Large files (up to 63-bit file length) are supported on file systems and operating systems that support large files.
There is a limit of (232)2 (1.844E+19) rows in a
MyISAM
table.The maximum number of indexes per
MyISAM
table is 64.The maximum number of columns per index is 16.
The maximum key length is 1000 bytes. This can also be changed by changing the source and recompiling. For the case of a key longer than 250 bytes, a larger key block size than the default of 1024 bytes is used.
When rows are inserted in sorted order (as when you are using an
AUTO_INCREMENT
column), the index tree is split so that the high node only contains one key. This improves space utilization in the index tree.Internal handling of one
AUTO_INCREMENT
column per table is supported.MyISAM
automatically updates this column forINSERT
andUPDATE
operations. This makesAUTO_INCREMENT
columns faster (at least 10%). Values at the top of the sequence are not reused after being deleted. (When anAUTO_INCREMENT
column is defined as the last column of a multiple-column index, reuse of values deleted from the top of a sequence does occur.) TheAUTO_INCREMENT
value can be reset withALTER TABLE
or myisamchk.Dynamic-sized rows are much less fragmented when mixing deletes with updates and inserts. This is done by automatically combining adjacent deleted blocks and by extending blocks if the next block is deleted.
MyISAM
supports concurrent inserts: If a table has no free blocks in the middle of the data file, you canINSERT
new rows into it at the same time that other threads are reading from the table. A free block can occur as a result of deleting rows or an update of a dynamic length row with more data than its current contents. When all free blocks are used up (filled in), future inserts become concurrent again. See Section 7.10.3, “Concurrent Inserts”.You can put the data file and index file in different directories on different physical devices to get more speed with the
DATA DIRECTORY
andINDEX DIRECTORY
table options toCREATE TABLE
. See Section 12.1.17, “CREATE TABLE
Синтаксис”.NULL
values are permitted in indexed columns. This takes 0 to 1 bytes per key.Each character column can have a different character set. See Section 9.1, “Character Set Support”.
There is a flag in the
MyISAM
index file that indicates whether the table was closed correctly. If mysqld is started with the--myisam-recover-options
option,MyISAM
tables are automatically checked when opened, and are repaired if the table wasn't closed properly.myisamchk marks tables as checked if you run it with the
--update-state
option. myisamchk --fast checks only those tables that don't have this mark.myisamchk --analyze stores statistics for portions of keys, as well as for entire keys.
myisampack can pack
BLOB
andVARCHAR
columns.
MyISAM
also supports the following features:
Additional Resources
A forum dedicated to the
MyISAM
storage engine is available at http://forums.mysql.com/list.php?21.
The following options to mysqld can be used to
change the behavior of MyISAM
tables. For
additional information, see Section 5.1.2, “Server Command Options”.
Table 13.11. MyISAM
Option/Variable
Reference
Name | Cmd-Line | Option file | System Var | Status Var | Var Scope | Dynamic |
---|---|---|---|---|---|---|
bulk_insert_buffer_size | Yes | Yes | Yes | Both | Yes | |
concurrent_insert | Yes | Yes | Yes | Global | Yes | |
delay-key-write | Yes | Yes | Global | Yes | ||
- Variable: delay_key_write | Yes | Global | Yes | |||
have_rtree_keys | Yes | Global | No | |||
key_buffer_size | Yes | Yes | Yes | Global | Yes | |
log-isam | Yes | Yes | ||||
myisam-block-size | Yes | Yes | ||||
myisam_data_pointer_size | Yes | Yes | Yes | Global | Yes | |
myisam_max_sort_file_size | Yes | Yes | Yes | Global | Yes | |
myisam_mmap_size | Yes | Yes | Yes | Global | No | |
myisam-recover | Yes | Yes | ||||
- Variable: myisam_recover_options | ||||||
myisam-recover-options | Yes | Yes | ||||
- Variable: myisam_recover_options | ||||||
myisam_recover_options | Yes | Global | No | |||
myisam_repair_threads | Yes | Yes | Yes | Both | Yes | |
myisam_sort_buffer_size | Yes | Yes | Yes | Both | Yes | |
myisam_stats_method | Yes | Yes | Yes | Both | Yes | |
myisam_use_mmap | Yes | Yes | Yes | Global | Yes | |
skip-concurrent-insert | Yes | Yes | ||||
- Variable: concurrent_insert | ||||||
tmp_table_size | Yes | Yes | Yes | Both | Yes |
Set the mode for automatic recovery of crashed
MyISAM
tables.Don't flush key buffers between writes for any
MyISAM
table.ЗамечаниеIf you do this, you should not access
MyISAM
tables from another program (such as from another MySQL server or with myisamchk) when the tables are in use. Doing so risks index corruption. Using--external-locking
does not eliminate this risk.
The following system variables affect the behavior of
MyISAM
tables. For additional information, see
Section 5.1.3, “Server System Variables”.
The size of the tree cache used in bulk insert optimization.
ЗамечаниеThis is a limit per thread!
The maximum size of the temporary file that MySQL is permitted to use while re-creating a
MyISAM
index (duringREPAIR TABLE
,ALTER TABLE
, orLOAD DATA INFILE
). If the file size would be larger than this value, the index is created using the key cache instead, which is slower. The value is given in bytes.Set the size of the buffer used when recovering tables.
Automatic recovery is activated if you start
mysqld with the
--myisam-recover-options
option. In
this case, when the server opens a MyISAM
table, it checks whether the table is marked as crashed or whether
the open count variable for the table is not 0 and you are running
the server with external locking disabled. If either of these
conditions is true, the following happens:
The server checks the table for errors.
If the server finds an error, it tries to do a fast table repair (with sorting and without re-creating the data file).
If the repair fails because of an error in the data file (for example, a duplicate-key error), the server tries again, this time re-creating the data file.
If the repair still fails, the server tries once more with the old repair option method (write row by row without sorting). This method should be able to repair any type of error and has low disk space requirements.
If the recovery wouldn't be able to recover all rows from
previously completed statements and you didn't specify
FORCE
in the value of the
--myisam-recover-options
option,
automatic repair aborts with an error message in the error log:
Error: Couldn't repair table: test.g00pages
If you specify FORCE
, a warning like this is
written instead:
Warning: Found 344 of 354 rows when repairing ./test/g00pages
Note that if the automatic recovery value includes
BACKUP
, the recovery process creates files with
names of the form
.
You should have a cron script that
automatically moves these files from the database directories to
backup media.
tbl_name-datetime
.BAK
MyISAM
tables use B-tree indexes. You can
roughly calculate the size for the index file as
(key_length+4)/0.67
, summed over all keys. This
is for the worst case when all keys are inserted in sorted order
and the table doesn't have any compressed keys.
String indexes are space compressed. If the first index part is a
string, it is also prefix compressed. Space compression makes the
index file smaller than the worst-case figure if a string column
has a lot of trailing space or is a
VARCHAR
column that is not always
used to the full length. Prefix compression is used on keys that
start with a string. Prefix compression helps if there are many
strings with an identical prefix.
In MyISAM
tables, you can also prefix compress
numbers by specifying the PACK_KEYS=1
table
option when you create the table. Numbers are stored with the high
byte first, so this helps when you have many integer keys that
have an identical prefix.
MyISAM
supports three different storage
formats. Two of them, fixed and dynamic format, are chosen
automatically depending on the type of columns you are using. The
third, compressed format, can be created only with the
myisampack utility (see
Section 4.6.5, “myisampack — Generate Compressed, Read-Only MyISAM Tables”).
When you use CREATE TABLE
or
ALTER TABLE
for a table that has no
BLOB
or
TEXT
columns, you can force the
table format to FIXED
or
DYNAMIC
with the ROW_FORMAT
table option.
See Section 12.1.17, “CREATE TABLE
Синтаксис”, for information about
ROW_FORMAT
.
You can decompress (unpack) compressed MyISAM
tables using myisamchk
--unpack
; see
Section 4.6.3, “myisamchk — MyISAM Table-Maintenance Utility”, for more information.
Static format is the default for MyISAM
tables. It is used when the table contains no variable-length
columns (VARCHAR
,
VARBINARY
,
BLOB
, or
TEXT
). Each row is stored using a
fixed number of bytes.
Of the three MyISAM
storage formats, static
format is the simplest and most secure (least subject to
corruption). It is also the fastest of the on-disk formats due
to the ease with which rows in the data file can be found on
disk: To look up a row based on a row number in the index,
multiply the row number by the row length to calculate the row
position. Also, when scanning a table, it is very easy to read a
constant number of rows with each disk read operation.
The security is evidenced if your computer crashes while the
MySQL server is writing to a fixed-format
MyISAM
file. In this case,
myisamchk can easily determine where each row
starts and ends, so it can usually reclaim all rows except the
partially written one. Note that MyISAM
table
indexes can always be reconstructed based on the data rows.
Fixed-length row format is only available for tables without
BLOB
or
TEXT
columns. Creating a table
with these columns with an explicit
ROW_FORMAT
clause will not raise an error
or warning; the format specification will be ignored.
Static-format tables have these characteristics:
CHAR
andVARCHAR
columns are space-padded to the specified column width, although the column type is not altered.BINARY
andVARBINARY
columns are padded with0x00
bytes to the column width.Very quick.
Easy to cache.
Easy to reconstruct after a crash, because rows are located in fixed positions.
Reorganization is unnecessary unless you delete a huge number of rows and want to return free disk space to the operating system. To do this, use
OPTIMIZE TABLE
or myisamchk -r.Usually require more disk space than dynamic-format tables.
Dynamic storage format is used if a MyISAM
table contains any variable-length columns
(VARCHAR
,
VARBINARY
,
BLOB
, or
TEXT
), or if the table was
created with the ROW_FORMAT=DYNAMIC
table
option.
Dynamic format is a little more complex than static format because each row has a header that indicates how long it is. A row can become fragmented (stored in noncontiguous pieces) when it is made longer as a result of an update.
You can use OPTIMIZE TABLE
or
myisamchk -r to defragment a table. If you
have fixed-length columns that you access or change frequently
in a table that also contains some variable-length columns, it
might be a good idea to move the variable-length columns to
other tables just to avoid fragmentation.
Dynamic-format tables have these characteristics:
All string columns are dynamic except those with a length less than four.
Each row is preceded by a bitmap that indicates which columns contain the empty string (for string columns) or zero (for numeric columns). Note that this does not include columns that contain
NULL
values. If a string column has a length of zero after trailing space removal, or a numeric column has a value of zero, it is marked in the bitmap and not saved to disk. Nonempty strings are saved as a length byte plus the string contents.Much less disk space usually is required than for fixed-length tables.
Each row uses only as much space as is required. However, if a row becomes larger, it is split into as many pieces as are required, resulting in row fragmentation. For example, if you update a row with information that extends the row length, the row becomes fragmented. In this case, you may have to run
OPTIMIZE TABLE
or myisamchk -r from time to time to improve performance. Use myisamchk -ei to obtain table statistics.More difficult than static-format tables to reconstruct after a crash, because rows may be fragmented into many pieces and links (fragments) may be missing.
The expected row length for dynamic-sized rows is calculated using the following expression:
3 + (
number of columns
+ 7) / 8 + (number of char columns
) + (packed size of numeric columns
) + (length of strings
) + (number of NULL columns
+ 7) / 8There is a penalty of 6 bytes for each link. A dynamic row is linked whenever an update causes an enlargement of the row. Each new link is at least 20 bytes, so the next enlargement probably goes in the same link. If not, another link is created. You can find the number of links using myisamchk -ed. All links may be removed with
OPTIMIZE TABLE
or myisamchk -r.
Compressed storage format is a read-only format that is generated with the myisampack tool. Compressed tables can be uncompressed with myisamchk.
Compressed tables have the following characteristics:
Compressed tables take very little disk space. This minimizes disk usage, which is helpful when using slow disks (such as CD-ROMs).
Each row is compressed separately, so there is very little access overhead. The header for a row takes up one to three bytes depending on the biggest row in the table. Each column is compressed differently. There is usually a different Huffman tree for each column. Some of the compression types are:
Suffix space compression.
Prefix space compression.
Numbers with a value of zero are stored using one bit.
If values in an integer column have a small range, the column is stored using the smallest possible type. For example, a
BIGINT
column (eight bytes) can be stored as aTINYINT
column (one byte) if all its values are in the range from-128
to127
.If a column has only a small set of possible values, the data type is converted to
ENUM
.A column may use any combination of the preceding compression types.
Can be used for fixed-length or dynamic-length rows.
While a compressed table is read only, and you cannot
therefore update or add rows in the table, DDL (Data
Definition Language) operations are still valid. For example,
you may still use DROP
to drop the table,
and TRUNCATE TABLE
to empty the
table.
The file format that MySQL uses to store data has been extensively tested, but there are always circumstances that may cause database tables to become corrupted. The following discussion describes how this can happen and how to handle it.
Even though the MyISAM
table format is very
reliable (all changes to a table made by an SQL statement are
written before the statement returns), you can still get
corrupted tables if any of the following events occur:
The mysqld process is killed in the middle of a write.
An unexpected computer shutdown occurs (for example, the computer is turned off).
Hardware failures.
You are using an external program (such as myisamchk) to modify a table that is being modified by the server at the same time.
A software bug in the MySQL or
MyISAM
code.
Typical symptoms of a corrupt table are:
You get the following error while selecting data from the table:
Incorrect key file for table: '...'. Try to repair it
Queries don't find rows in the table or return incomplete results.
You can check the health of a MyISAM
table
using the CHECK TABLE
statement,
and repair a corrupted MyISAM
table with
REPAIR TABLE
. When
mysqld is not running, you can also check or
repair a table with the myisamchk command.
See Section 12.7.2.2, “CHECK TABLE
Синтаксис”,
Section 12.7.2.5, “REPAIR TABLE
Синтаксис”, and Section 4.6.3, “myisamchk — MyISAM Table-Maintenance Utility”.
If your tables become corrupted frequently, you should try to
determine why this is happening. The most important thing to
know is whether the table became corrupted as a result of a
server crash. You can verify this easily by looking for a recent
restarted mysqld
message in the error log. If
there is such a message, it is likely that table corruption is a
result of the server dying. Otherwise, corruption may have
occurred during normal operation. This is a bug. You should try
to create a reproducible test case that demonstrates the
problem. See Section C.5.4.2, “What to Do If MySQL Keeps Crashing”, and
MySQL
Internals: Porting.
Each MyISAM
index file
(.MYI
file) has a counter in the header
that can be used to check whether a table has been closed
properly. If you get the following warning from
CHECK TABLE
or
myisamchk, it means that this counter has
gone out of sync:
clients are using or haven't closed the table properly
This warning doesn't necessarily mean that the table is corrupted, but you should at least check the table.
The counter works as follows:
The first time a table is updated in MySQL, a counter in the header of the index files is incremented.
The counter is not changed during further updates.
When the last instance of a table is closed (because a
FLUSH TABLES
operation was performed or because there is no room in the table cache), the counter is decremented if the table has been updated at any point.When you repair the table or check the table and it is found to be okay, the counter is reset to zero.
To avoid problems with interaction with other processes that might check the table, the counter is not decremented on close if it was zero.
In other words, the counter can become incorrect only under these conditions:
A
MyISAM
table is copied without first issuingLOCK TABLES
andFLUSH TABLES
.MySQL has crashed between an update and the final close. (Note that the table may still be okay, because MySQL always issues writes for everything between each statement.)
A table was modified by myisamchk --recover or myisamchk --update-state at the same time that it was in use by mysqld.
Multiple mysqld servers are using the table and one server performed a
REPAIR TABLE
orCHECK TABLE
on the table while it was in use by another server. In this setup, it is safe to useCHECK TABLE
, although you might get the warning from other servers. However,REPAIR TABLE
should be avoided because when one server replaces the data file with a new one, this is not known to the other servers.In general, it is a bad idea to share a data directory among multiple servers. See Section 5.6, “Running Multiple MySQL Instances on One Machine”, for additional discussion.
The MEMORY
storage engine (formerly known as
HEAP
) creates special-purpose tables with
contents that are stored in memory. Because the data is vulnerable
to crashes, hardware issues, or power outages, only use these tables
as temporary work areas or read-only caches for data pulled from
other tables.
Table 13.12. MEMORY
Storage Engine
Features
Storage limits | RAM | Transactions | No | Locking granularity | Table |
MVCC | No | Geospatial data type support | No | Geospatial indexing support | No |
B-tree indexes | Yes | Hash indexes | Yes | Full-text search indexes | No |
Clustered indexes | No | Data caches | N/A | Index caches | N/A |
Compressed data | No | Encrypted data[a] | Yes | Cluster database support | No |
Replication support[b] | Yes | Foreign key support | No | Backup / point-in-time recovery[c] | Yes |
Query cache support | Yes | Update statistics for data dictionary | Yes | ||
[a] Implemented in the server (via encryption functions), rather than in the storage engine. [b] Implemented in the server, rather than in the storage product. [c] Implemented in the server, rather than in the storage product. |
When to Use MEMORY
or MySQL Cluster.
Developers looking to deploy applications that use the
MEMORY
storage engine for important, highly
available, or frequently updated data should consider whether
MySQL Cluster is a better choice. A typical use case for the
MEMORY
engine involves these characteristics:
Operations involving transient, non-critical data such as session management or caching. When the MySQL server halts or restarts, the data in
MEMORY
tables is lost.In-memory storage for fast access and low latency. Data volume can fit entirely in memory without causing the operating system to swap out virtual memory pages.
A read-only or read-mostly data access pattern (limited updates).
MySQL Cluster offers the same features as the
MEMORY
engine with higher performance levels, and
provides additional features not available with
MEMORY
:
Row-level locking and multiple-thread operation for low contention between clients.
Scalability even with statement mixes that include writes.
Optional disk-backed operation for data durability.
Shared-nothing architecture and multiple-host operation with no single point of failure, enabling 99.999% availability.
Automatic data distribution across nodes; application developers need not craft custom sharding or partitioning solutions.
Support for variable-length data types (including
BLOB
andTEXT
) not supported byMEMORY
.
For a white paper with more detailed comparison of the
MEMORY
storage engine and MySQL Cluster, see
Scaling
Web Services with MySQL Cluster: An Alternative to the MySQL Memory
Storage Engine. This white paper includes a performance
study of the two technologies and a step-by-step guide describing
how existing MEMORY
users can migrate to MySQL
Cluster.
Performance Characteristics
MEMORY
performance is constrained by contention
resulting from single-thread execution and table lock overhead when
processing updates. This limits scalability when load increases,
particularly for statement mixes that include writes.
Despite the in-memory processing for MEMORY
tables, they are not necessarily faster than
InnoDB
tables on a busy server, for
general-purpose queries, or under a read/write workload. In
particular, the table locking involved with performing updates can
slow down concurrent usage of MEMORY
tables from
multiple sessions.
Depending on the kinds of queries performed on a
MEMORY
table, you might create indexes as either
the default hash data structure (for looking up single values based
on a unique key), or a general-purpose B-tree data structure (for
all kinds of queries involving equality, inequality, or range
operators such as less than or greater than). The following sections
illustrate the syntax for creating both kinds of indexes. A common
performance issue is using the default hash indexes in workloads
where B-tree indexes are more efficient.
Physical Characteristics of MEMORY
Tables
The MEMORY
storage engine associates each table
with one disk file, which stores the table definition (not the
data). The file name begins with the table name and has an extension
of .frm
.
MEMORY
tables have the following characteristics:
Space for
MEMORY
tables is allocated in small blocks. Tables use 100% dynamic hashing for inserts. No overflow area or extra key space is needed. No extra space is needed for free lists. Deleted rows are put in a linked list and are reused when you insert new data into the table.MEMORY
tables also have none of the problems commonly associated with deletes plus inserts in hashed tables.MEMORY
tables use a fixed-length row-storage format. Variable-length types such asVARCHAR
are stored using a fixed length.MEMORY
includes support forAUTO_INCREMENT
columns.Non-
TEMPORARY
MEMORY
tables are shared among all clients, just like any other non-TEMPORARY
table.
DDL Operations for MEMORY
Tables
To create a MEMORY
table, specify the clause
ENGINE=MEMORY
on the CREATE
TABLE
statement.
CREATE TABLE t (i INT) ENGINE = MEMORY;
As indicated by the engine name, MEMORY
tables
are stored in memory. They use hash indexes by default, which makes
them very fast for single-value lookups, and very useful for
creating temporary tables. However, when the server shuts down, all
rows stored in MEMORY
tables are lost. The tables
themselves continue to exist because their definitions are stored in
.frm
files on disk, but they are empty when the
server restarts.
This example shows how you might create, use, and remove a
MEMORY
table:
mysql>CREATE TABLE test ENGINE=MEMORY
->SELECT ip,SUM(downloads) AS down
->FROM log_table GROUP BY ip;
mysql>SELECT COUNT(ip),AVG(down) FROM test;
mysql>DROP TABLE test;
The maximum size of MEMORY
tables is limited by
the max_heap_table_size
system
variable, which has a default value of 16MB. To enforce different
size limits for MEMORY
tables, change the value
of this variable. The value in effect for
CREATE TABLE
, or a subsequent
ALTER TABLE
or
TRUNCATE TABLE
, is the value used for
the life of the table. A server restart also sets the maximum size
of existing MEMORY
tables to the global
max_heap_table_size
value. You can
set the size for individual tables as described later in this
section.
Indexes
The MEMORY
storage engine supports both
HASH
and BTREE
indexes. You
can specify one or the other for a given index by adding a
USING
clause as shown here:
CREATE TABLE lookup (id INT, INDEX USING HASH (id)) ENGINE = MEMORY; CREATE TABLE lookup (id INT, INDEX USING BTREE (id)) ENGINE = MEMORY;
For general characteristics of B-tree and hash indexes, see Section 7.3.1, “How MySQL Uses Indexes”.
MEMORY
tables can have up to 64 indexes per
table, 16 columns per index and a maximum key length of 3072 bytes.
If a MEMORY
table hash index has a high degree of
key duplication (many index entries containing the same value),
updates to the table that affect key values and all deletes are
significantly slower. The degree of this slowdown is proportional to
the degree of duplication (or, inversely proportional to the index
cardinality). You can use a BTREE
index to avoid
this problem.
MEMORY
tables can have nonunique keys. (This is
an uncommon feature for implementations of hash indexes.)
Columns that are indexed can contain NULL
values.
User-Created and Temporary Tables
MEMORY
table contents are stored in memory, which
is a property that MEMORY
tables share with
internal temporary tables that the server creates on the fly while
processing queries. However, the two types of tables differ in that
MEMORY
tables are not subject to storage
conversion, whereas internal temporary tables are:
If an internal temporary table becomes too large, the server automatically converts it to on-disk storage, as described in Section 7.4.3.3, “How MySQL Uses Internal Temporary Tables”.
User-created
MEMORY
tables are never converted to disk tables.
Loading Data
To populate a MEMORY
table when the MySQL server
starts, you can use the --init-file
option. For example, you can put statements such as
INSERT INTO ...
SELECT
or LOAD
DATA INFILE
into this file to load the table from a
persistent data source. See Section 5.1.2, “Server Command Options”, and
Section 12.2.6, “LOAD DATA INFILE
Синтаксис”.
For loading data into MEMORY
tables accessed by
other sessions concurrently, MEMORY
supports
INSERT DELAYED
. See
Section 12.2.5.2, “INSERT DELAYED
Синтаксис”.
MEMORY
Tables and Replication
A server's MEMORY
tables become empty when it is
shut down and restarted. If the server is a replication master, its
slaves are not aware that these tables have become empty, so you see
out-of-date content if you select data from the tables on the
slaves. To synchronize master and slave MEMORY
tables, when a MEMORY
table is used on a master
for the first time since it was started, a
DELETE
statement is written to the
master's binary log, to empty the table on the slaves also. The
slave still has outdated data in the table during the interval
between the master's restart and its first use of the table. To
avoid this interval when a direct query to the slave could return
stale data, use the --init-file
option to populate the MEMORY
table on the master
at startup.
Managing Memory Use
The server needs sufficient memory to maintain all
MEMORY
tables that are in use at the same time.
Memory is not reclaimed if you delete individual rows from a
MEMORY
table. Memory is reclaimed only when the
entire table is deleted. Memory that was previously used for deleted
rows is re-used for new rows within the same table. To free all the
memory used by a MEMORY
table when you no longer
require its contents, execute DELETE
or TRUNCATE TABLE
to remove all rows,
or remove the table altogether using DROP
TABLE
. To free up the memory used by deleted rows, use
ALTER TABLE ENGINE=MEMORY
to force a table
rebuild.
The memory needed for one row in a MEMORY
table
is calculated using the following expression:
SUM_OVER_ALL_BTREE_KEYS(max_length_of_key
+ sizeof(char*) * 4) + SUM_OVER_ALL_HASH_KEYS(sizeof(char*) * 2) + ALIGN(length_of_row
+1, sizeof(char*))
ALIGN()
represents a round-up factor to cause the
row length to be an exact multiple of the char
pointer size. sizeof(char*)
is 4 on 32-bit
machines and 8 on 64-bit machines.
As mentioned earlier, the
max_heap_table_size
system variable
sets the limit on the maximum size of MEMORY
tables. To control the maximum size for individual tables, set the
session value of this variable before creating each table. (Do not
change the global
max_heap_table_size
value unless
you intend the value to be used for MEMORY
tables
created by all clients.) The following example creates two
MEMORY
tables, with a maximum size of 1MB and
2MB, respectively:
mysql>SET max_heap_table_size = 1024*1024;
Query OK, 0 rows affected (0.00 sec) mysql>CREATE TABLE t1 (id INT, UNIQUE(id)) ENGINE = MEMORY;
Query OK, 0 rows affected (0.01 sec) mysql>SET max_heap_table_size = 1024*1024*2;
Query OK, 0 rows affected (0.00 sec) mysql>CREATE TABLE t2 (id INT, UNIQUE(id)) ENGINE = MEMORY;
Query OK, 0 rows affected (0.00 sec)
Both tables revert to the server's global
max_heap_table_size
value if the
server restarts.
You can also specify a MAX_ROWS
table option in
CREATE TABLE
statements for
MEMORY
tables to provide a hint about the number
of rows you plan to store in them. This does not enable the table to
grow beyond the max_heap_table_size
value, which still acts as a constraint on maximum table size. For
maximum flexibility in being able to use
MAX_ROWS
, set
max_heap_table_size
at least as
high as the value to which you want each MEMORY
table to be able to grow.
Additional Resources
A forum dedicated to the MEMORY
storage engine is
available at http://forums.mysql.com/list.php?92.
The CSV
storage engine stores data in text files
using comma-separated values format.
The CSV
storage engine is always compiled into
the MySQL server.
To examine the source for the CSV
engine, look in
the storage/csv
directory of a MySQL source
distribution.
When you create a CSV
table, the server creates a
table format file in the database directory. The file begins with
the table name and has an .frm
extension. The
storage engine also creates a data file. Its name begins with the
table name and has a .CSV
extension. The data
file is a plain text file. When you store data into the table, the
storage engine saves it into the data file in comma-separated values
format.
mysql>CREATE TABLE test (i INT NOT NULL, c CHAR(10) NOT NULL)
->ENGINE = CSV;
Query OK, 0 rows affected (0.12 sec) mysql>INSERT INTO test VALUES(1,'record one'),(2,'record two');
Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql>SELECT * FROM test;
+------+------------+ | i | c | +------+------------+ | 1 | record one | | 2 | record two | +------+------------+ 2 rows in set (0.00 sec)
Creating a CSV table also creates a corresponding Metafile that
stores the state of the table and the number of rows that exist in
the table. The name of this file is the same as the name of the
table with the extension CSM
.
If you examine the test.CSV
file in the
database directory created by executing the preceding statements,
its contents should look like this:
"1","record one" "2","record two"
This format can be read, and even written, by spreadsheet applications such as Microsoft Excel or StarOffice Calc.
The CSV storage engines supports the CHECK
and
REPAIR
statements to verify and if possible
repair a damaged CSV table.
When running the CHECK
statement, the CSV file
will be checked for validity by looking for the correct field
separators, escaped fields (matching or missing quotation marks),
the correct number of fields compared to the table definition and
the existence of a corresponding CSV metafile. The first invalid
row discovered will report an error. Checking a valid table
produces output like that shown below:
mysql> check table csvtest;
+--------------+-------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------+-------+----------+----------+
| test.csvtest | check | status | OK |
+--------------+-------+----------+----------+
1 row in set (0.00 sec)
A check on a corrupted table returns a fault:
mysql> check table csvtest;
+--------------+-------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------+-------+----------+----------+
| test.csvtest | check | error | Corrupt |
+--------------+-------+----------+----------+
1 row in set (0.01 sec)
If the check fails, the table is marked as crashed (corrupt). Once
a table has been marked as corrupt, it is automatically repaired
when you next run CHECK
or execute a
SELECT
statement. The corresponding
corrupt status and new status will be displayed when running
CHECK
:
mysql> check table csvtest;
+--------------+-------+----------+----------------------------+
| Table | Op | Msg_type | Msg_text |
+--------------+-------+----------+----------------------------+
| test.csvtest | check | warning | Table is marked as crashed |
| test.csvtest | check | status | OK |
+--------------+-------+----------+----------------------------+
2 rows in set (0.08 sec)
To repair a table you can use REPAIR
, this
copies as many valid rows from the existing CSV data as possible,
and then replaces the existing CSV file with the recovered rows.
Any rows beyond the corrupted data are lost.
mysql> repair table csvtest;
+--------------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------+--------+----------+----------+
| test.csvtest | repair | status | OK |
+--------------+--------+----------+----------+
1 row in set (0.02 sec)
Note that during repair, only the rows from the CSV file up to the first damaged row are copied to the new table. All other rows from the first damaged row to the end of the table are removed, even valid rows.
The CSV
storage engine does not support
indexing.
Partitioning is not supported for tables using the
CSV
storage engine.
Tables using the CSV
storage engine cannot be
created with NULL
columns. However, for
backward compatibility, you can continue to use such tables that
were created in previous MySQL releases. (Bug #32050)
The ARCHIVE
storage engine is used for storing
large amounts of data without indexes in a very small footprint.
Table 13.13. ARCHIVE
Storage Engine
Features
Storage limits | None | Transactions | No | Locking granularity | Table |
MVCC | No | Geospatial data type support | Yes | Geospatial indexing support | No |
B-tree indexes | No | Hash indexes | No | Full-text search indexes | No |
Clustered indexes | No | Data caches | No | Index caches | No |
Compressed data | Yes | Encrypted data[a] | Yes | Cluster database support | No |
Replication support[b] | Yes | Foreign key support | No | Backup / point-in-time recovery[c] | Yes |
Query cache support | Yes | Update statistics for data dictionary | Yes | ||
[a] Implemented in the server (via encryption functions), rather than in the storage engine. [b] Implemented in the server, rather than in the storage product. [c] Implemented in the server, rather than in the storage product. |
The ARCHIVE
storage engine is included in MySQL
binary distributions. To enable this storage engine if you build
MySQL from source, invoke CMake with the
-DWITH_ARCHIVE_STORAGE_ENGINE
option.
To examine the source for the ARCHIVE
engine,
look in the storage/archive
directory of a
MySQL source distribution.
You can check whether the ARCHIVE
storage engine
is available with the SHOW ENGINES
statement.
When you create an ARCHIVE
table, the server
creates a table format file in the database directory. The file
begins with the table name and has an .frm
extension. The storage engine creates other files, all having names
beginning with the table name. The data file has an extension of
.ARZ
. An .ARN
file may
appear during optimization operations.
The ARCHIVE
engine supports
INSERT
and
SELECT
, but not
DELETE
,
REPLACE
, or
UPDATE
. It does support
ORDER BY
operations,
BLOB
columns, and basically all but
spatial data types (see Section 11.17.4.1, “MySQL Spatial Data Types”).
The ARCHIVE
engine uses row-level locking.
The ARCHIVE
engine supports the
AUTO_INCREMENT
column attribute. The
AUTO_INCREMENT
column can have either a unique or
nonunique index. Attempting to create an index on any other column
results in an error. The ARCHIVE
engine also
supports the AUTO_INCREMENT
table option in
CREATE TABLE
and
ALTER TABLE
statements to specify the
initial sequence value for a new table or reset the sequence value
for an existing table, respectively.
The ARCHIVE
engine ignores
BLOB
columns if they are not
requested and scans past them while reading.
Storage: Rows are compressed as
they are inserted. The ARCHIVE
engine uses
zlib
lossless data compression (see
http://www.zlib.net/). You can use
OPTIMIZE TABLE
to analyze the table
and pack it into a smaller format (for a reason to use
OPTIMIZE TABLE
, see later in this
section). The engine also supports CHECK
TABLE
. There are several types of insertions that are
used:
An
INSERT
statement just pushes rows into a compression buffer, and that buffer flushes as necessary. The insertion into the buffer is protected by a lock. ASELECT
forces a flush to occur, unless the only insertions that have come in wereINSERT DELAYED
(those flush as necessary). See Section 12.2.5.2, “INSERT DELAYED
Синтаксис”.A bulk insert is visible only after it completes, unless other inserts occur at the same time, in which case it can be seen partially. A
SELECT
never causes a flush of a bulk insert unless a normal insert occurs while it is loading.
Retrieval: On retrieval, rows are
uncompressed on demand; there is no row cache. A
SELECT
operation performs a complete
table scan: When a SELECT
occurs, it
finds out how many rows are currently available and reads that
number of rows. SELECT
is performed
as a consistent read. Note that lots of
SELECT
statements during insertion
can deteriorate the compression, unless only bulk or delayed inserts
are used. To achieve better compression, you can use
OPTIMIZE TABLE
or
REPAIR TABLE
. The number of rows in
ARCHIVE
tables reported by
SHOW TABLE STATUS
is always accurate.
See Section 12.7.2.4, “OPTIMIZE TABLE
Синтаксис”,
Section 12.7.2.5, “REPAIR TABLE
Синтаксис”, and
Section 12.7.5.37, “SHOW TABLE STATUS
Синтаксис”.
Additional Resources
A forum dedicated to the
ARCHIVE
storage engine is available at http://forums.mysql.com/list.php?112.
The BLACKHOLE
storage engine acts as a
“black hole” that accepts data but throws it away and
does not store it. Retrievals always return an empty result:
mysql>CREATE TABLE test(i INT, c CHAR(10)) ENGINE = BLACKHOLE;
Query OK, 0 rows affected (0.03 sec) mysql>INSERT INTO test VALUES(1,'record one'),(2,'record two');
Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql>SELECT * FROM test;
Empty set (0.00 sec)
To enable the BLACKHOLE
storage engine if you
build MySQL from source, invoke CMake with the
-DWITH_BLACKHOLE_STORAGE_ENGINE
option.
To examine the source for the BLACKHOLE
engine,
look in the sql
directory of a MySQL source
distribution.
When you create a BLACKHOLE
table, the server
creates a table format file in the database directory. The file
begins with the table name and has an .frm
extension. There are no other files associated with the table.
The BLACKHOLE
storage engine supports all kinds
of indexes. That is, you can include index declarations in the table
definition.
You can check whether the BLACKHOLE
storage
engine is available with the SHOW
ENGINES
statement.
Inserts into a BLACKHOLE
table do not store any
data, but if the binary log is enabled, the SQL statements are
logged (and replicated to slave servers). This can be useful as a
repeater or filter mechanism. Suppose that your application requires
slave-side filtering rules, but transferring all binary log data to
the slave first results in too much traffic. In such a case, it is
possible to set up on the master host a “dummy” slave
process whose default storage engine is
BLACKHOLE
, depicted as follows:
The master writes to its binary log. The “dummy”
mysqld process acts as a slave, applying the
desired combination of replicate-do-*
and
replicate-ignore-*
rules, and writes a new,
filtered binary log of its own. (See
Section 15.1.3, “Replication and Binary Logging Options and Variables”.) This filtered log is
provided to the slave.
The dummy process does not actually store any data, so there is little processing overhead incurred by running the additional mysqld process on the replication master host. This type of setup can be repeated with additional replication slaves.
INSERT
triggers for
BLACKHOLE
tables work as expected. However,
because the BLACKHOLE
table does not actually
store any data, UPDATE
and
DELETE
triggers are not activated:
The FOR EACH ROW
clause in the trigger definition
does not apply because there are no rows.
Other possible uses for the BLACKHOLE
storage
engine include:
Verification of dump file syntax.
Measurement of the overhead from binary logging, by comparing performance using
BLACKHOLE
with and without binary logging enabled.BLACKHOLE
is essentially a “no-op” storage engine, so it could be used for finding performance bottlenecks not related to the storage engine itself.
The BLACKHOLE
engine is transaction-aware, in the
sense that committed transactions are written to the binary log and
rolled-back transactions are not.
Blackhole Engine and Auto Increment Columns
The Blackhole engine is a no-op engine. Any operations performed on a table using Blackhole will have no effect. This should be born in mind when considering the behavior of primary key columns that auto increment. The engine will not automatically increment field values, and does not retain auto increment field state. This has important implications in replication.
Consider the following replication scenario where all three of the following conditions apply:
On a master server there is a blackhole table with an auto increment field that is a primary key.
On a slave the same table exists but using the MyISAM engine.
Inserts are performed into the master's table without explicitly setting the auto increment value in the
INSERT
statement itself or through using aSET INSERT_ID
statement.
In this scenario replication will fail with a duplicate entry error on the primary key column.
In statement based replication, the value of
INSERT_ID
in the context event will always be the
same. Replication will therefore fail due to trying insert a row
with a duplicate value for a primary key column.
In row based replication, the value that the engine returns for the row always be the same for each insert. This will result in the slave attempting to replay two insert log entries using the same value for the primary key column, and so replication will fail.
Column Filtering
When using row-based replication,
(binlog_format=ROW
), a slave where
the last columns are missing from a table is supported, as described
in the section
Section 15.4.1.6, “Replication with Differing Table Definitions on Master and Slave”.
This filtering works on the slave side, that is, the columns are copied to the slave before they are filtered out. There are at least two cases where it is not desirable to copy the columns to the slave:
If the data is confidential, so the slave server should not have access to it.
If the master has many slaves, filtering before sending to the slaves may reduce network traffic.
Master column filtering can be achieved using the
BLACKHOLE
engine. This is carried out in a way
similar to how master table filtering is achieved - by using the
BLACKHOLE
engine and the option
--replicate-[do|ignore]-table
.
The setup for the master is:
CREATE TABLE t1 (public_col_1, ..., public_col_N, secret_col_1, ..., secret_col_M) ENGINE=MyISAM;
The setup for the trusted slave is:
CREATE TABLE t1 (public_col_1, ..., public_col_N) ENGINE=BLACKHOLE;
The setup for the untrusted slave is:
CREATE TABLE t1 (public_col_1, ..., public_col_N) ENGINE=MyISAM;
The MERGE
storage engine, also known as the
MRG_MyISAM
engine, is a collection of identical
MyISAM
tables that can be used as one.
“Identical” means that all tables have identical column
and index information. You cannot merge MyISAM
tables in which the columns are listed in a different order, do not
have exactly the same columns, or have the indexes in different
order. However, any or all of the MyISAM
tables
can be compressed with myisampack. See
Section 4.6.5, “myisampack — Generate Compressed, Read-Only MyISAM Tables”. Differences in table options such as
AVG_ROW_LENGTH
, MAX_ROWS
, or
PACK_KEYS
do not matter.
An alternative to a MERGE
table is a partitioned
table, which stores partitions of a single table in separate files.
Partitioning enables some operations to be performed more
efficiently and is not limited to the MyISAM
storage engine. For more information, see
Глава 17, Partitioning.
When you create a MERGE
table, MySQL creates two
files on disk. The files have names that begin with the table name
and have an extension to indicate the file type. An
.frm
file stores the table format, and an
.MRG
file contains the names of the underlying
MyISAM
tables that should be used as one. The
tables do not have to be in the same database as the
MERGE
table.
You can use SELECT
,
DELETE
,
UPDATE
, and
INSERT
on MERGE
tables. You must have SELECT
,
DELETE
, and
UPDATE
privileges on the
MyISAM
tables that you map to a
MERGE
table.
The use of MERGE
tables entails the following
security issue: If a user has access to MyISAM
table t
, that user can create a
MERGE
table m
that
accesses t
. However, if the user's
privileges on t
are subsequently
revoked, the user can continue to access
t
by doing so through
m
.
Use of DROP TABLE
with a
MERGE
table drops only the
MERGE
specification. The underlying tables are
not affected.
To create a MERGE
table, you must specify a
UNION=(
option that indicates which list-of-tables
)MyISAM
tables to use.
You can optionally specify an INSERT_METHOD
option to control how inserts into the MERGE
table take place. Use a value of FIRST
or
LAST
to cause inserts to be made in the first or
last underlying table, respectively. If you specify no
INSERT_METHOD
option or if you specify it with a
value of NO
, inserts into the
MERGE
table are not permitted and attempts to do
so result in an error.
The following example shows how to create a MERGE
table:
mysql>CREATE TABLE t1 (
->a INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
->message CHAR(20)) ENGINE=MyISAM;
mysql>CREATE TABLE t2 (
->a INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
->message CHAR(20)) ENGINE=MyISAM;
mysql>INSERT INTO t1 (message) VALUES ('Testing'),('table'),('t1');
mysql>INSERT INTO t2 (message) VALUES ('Testing'),('table'),('t2');
mysql>CREATE TABLE total (
->a INT NOT NULL AUTO_INCREMENT,
->message CHAR(20), INDEX(a))
->ENGINE=MERGE UNION=(t1,t2) INSERT_METHOD=LAST;
Note that column a
is indexed as a
PRIMARY KEY
in the underlying
MyISAM
tables, but not in the
MERGE
table. There it is indexed but not as a
PRIMARY KEY
because a MERGE
table cannot enforce uniqueness over the set of underlying tables.
(Similarly, a column with a UNIQUE
index in the
underlying tables should be indexed in the MERGE
table but not as a UNIQUE
index.)
After creating the MERGE
table, you can use it to
issue queries that operate on the group of tables as a whole:
mysql> SELECT * FROM total;
+---+---------+
| a | message |
+---+---------+
| 1 | Testing |
| 2 | table |
| 3 | t1 |
| 1 | Testing |
| 2 | table |
| 3 | t2 |
+---+---------+
To remap a MERGE
table to a different collection
of MyISAM
tables, you can use one of the
following methods:
DROP
theMERGE
table and re-create it.Use
ALTER TABLE
to change the list of underlying tables.tbl_name
UNION=(...)It is also possible to use
ALTER TABLE ... UNION=()
(that is, with an emptyUNION
clause) to remove all of the underlying tables.
The underlying table definitions and indexes must conform closely to
the definition of the MERGE
table. Conformance is
checked when a table that is part of a MERGE
table is opened, not when the MERGE
table is
created. If any table fails the conformance checks, the operation
that triggered the opening of the table fails. This means that
changes to the definitions of tables within a
MERGE
may cause a failure when the
MERGE
table is accessed. The conformance checks
applied to each table are:
The underlying table and the
MERGE
table must have the same number of columns.The column order in the underlying table and the
MERGE
table must match.Additionally, the specification for each corresponding column in the parent
MERGE
table and the underlying tables are compared and must satisfy these checks:The column type in the underlying table and the
MERGE
table must be equal.The column length in the underlying table and the
MERGE
table must be equal.The column of the underlying table and the
MERGE
table can beNULL
.
The underlying table must have at least as many indexes as the
MERGE
table. The underlying table may have more indexes than theMERGE
table, but cannot have fewer.ЗамечаниеA known issue exists where indexes on the same columns must be in identical order, in both the
MERGE
table and the underlyingMyISAM
table. See Bug #33653.Each index must satisfy these checks:
The index type of the underlying table and the
MERGE
table must be the same.The number of index parts (that is, multiple columns within a compound index) in the index definition for the underlying table and the
MERGE
table must be the same.For each index part:
Index part lengths must be equal.
Index part types must be equal.
Index part languages must be equal.
Check whether index parts can be
NULL
.
If a MERGE
table cannot be opened or used because
of a problem with an underlying table, CHECK
TABLE
displays information about which table caused the
problem.
Additional Resources
A forum dedicated to the
MERGE
storage engine is available at http://forums.mysql.com/list.php?93.
MERGE
tables can help you solve the following
problems:
Easily manage a set of log tables. For example, you can put data from different months into separate tables, compress some of them with myisampack, and then create a
MERGE
table to use them as one.Obtain more speed. You can split a large read-only table based on some criteria, and then put individual tables on different disks. A
MERGE
table structured this way could be much faster than using a single large table.Perform more efficient searches. If you know exactly what you are looking for, you can search in just one of the underlying tables for some queries and use a
MERGE
table for others. You can even have many differentMERGE
tables that use overlapping sets of tables.Perform more efficient repairs. It is easier to repair individual smaller tables that are mapped to a
MERGE
table than to repair a single large table.Instantly map many tables as one. A
MERGE
table need not maintain an index of its own because it uses the indexes of the individual tables. As a result,MERGE
table collections are very fast to create or remap. (You must still specify the index definitions when you create aMERGE
table, even though no indexes are created.)If you have a set of tables from which you create a large table on demand, you can instead create a
MERGE
table from them on demand. This is much faster and saves a lot of disk space.Exceed the file size limit for the operating system. Each
MyISAM
table is bound by this limit, but a collection ofMyISAM
tables is not.You can create an alias or synonym for a
MyISAM
table by defining aMERGE
table that maps to that single table. There should be no really notable performance impact from doing this (only a couple of indirect calls andmemcpy()
calls for each read).
The disadvantages of MERGE
tables are:
You can use only identical
MyISAM
tables for aMERGE
table.Some
MyISAM
features are unavailable inMERGE
tables. For example, you cannot createFULLTEXT
indexes onMERGE
tables. (You can createFULLTEXT
indexes on the underlyingMyISAM
tables, but you cannot search theMERGE
table with a full-text search.)If the
MERGE
table is nontemporary, all underlyingMyISAM
tables must be nontemporary. If theMERGE
table is temporary, theMyISAM
tables can be any mix of temporary and nontemporary.MERGE
tables use more file descriptors thanMyISAM
tables. If 10 clients are using aMERGE
table that maps to 10 tables, the server uses (10 × 10) + 10 file descriptors. (10 data file descriptors for each of the 10 clients, and 10 index file descriptors shared among the clients.)Index reads are slower. When you read an index, the
MERGE
storage engine needs to issue a read on all underlying tables to check which one most closely matches a given index value. To read the next index value, theMERGE
storage engine needs to search the read buffers to find the next value. Only when one index buffer is used up does the storage engine need to read the next index block. This makesMERGE
indexes much slower oneq_ref
searches, but not much slower onref
searches. For more information abouteq_ref
andref
, see Section 12.8.2, “EXPLAIN
Синтаксис”.
The following are known problems with MERGE
tables:
In versions of MySQL Server prior to 5.1.23, it was possible to create temporary merge tables with nontemporary child MyISAM tables.
From versions 5.1.23, MERGE children were locked through the parent table. If the parent was temporary, it was not locked and so the children were not locked either. Parallel use of the MyISAM tables corrupted them.
If you use
ALTER TABLE
to change aMERGE
table to another storage engine, the mapping to the underlying tables is lost. Instead, the rows from the underlyingMyISAM
tables are copied into the altered table, which then uses the specified storage engine.The
INSERT_METHOD
table option for aMERGE
table indicates which underlyingMyISAM
table to use for inserts into theMERGE
table. However, use of theAUTO_INCREMENT
table option for thatMyISAM
table has no effect for inserts into theMERGE
table until at least one row has been inserted directly into theMyISAM
table.A
MERGE
table cannot maintain uniqueness constraints over the entire table. When you perform anINSERT
, the data goes into the first or lastMyISAM
table (as determined by theINSERT_METHOD
option). MySQL ensures that unique key values remain unique within thatMyISAM
table, but not over all the underlying tables in the collection.Because the
MERGE
engine cannot enforce uniqueness over the set of underlying tables,REPLACE
does not work as expected. The two key facts are:REPLACE
can detect unique key violations only in the underlying table to which it is going to write (which is determined by theINSERT_METHOD
option). This differs from violations in theMERGE
table itself.If
REPLACE
detects a unique key violation, it will change only the corresponding row in the underlying table it is writing to; that is, the first or last table, as determined by theINSERT_METHOD
option.
Similar considerations apply for
INSERT ... ON DUPLICATE KEY UPDATE
.MERGE
tables do not support partitioning. That is, you cannot partition aMERGE
table, nor can any of aMERGE
table's underlyingMyISAM
tables be partitioned.You should not use
ANALYZE TABLE
,REPAIR TABLE
,OPTIMIZE TABLE
,ALTER TABLE
,DROP TABLE
,DELETE
without aWHERE
clause, orTRUNCATE TABLE
on any of the tables that are mapped into an openMERGE
table. If you do so, theMERGE
table may still refer to the original table and yield unexpected results. To work around this problem, ensure that noMERGE
tables remain open by issuing aFLUSH TABLES
statement prior to performing any of the named operations.The unexpected results include the possibility that the operation on the
MERGE
table will report table corruption. If this occurs after one of the named operations on the underlyingMyISAM
tables, the corruption message is spurious. To deal with this, issue aFLUSH TABLES
statement after modifying theMyISAM
tables.DROP TABLE
on a table that is in use by aMERGE
table does not work on Windows because theMERGE
storage engine's table mapping is hidden from the upper layer of MySQL. Windows does not permit open files to be deleted, so you first must flush allMERGE
tables (withFLUSH TABLES
) or drop theMERGE
table before dropping the table.The definition of the
MyISAM
tables and theMERGE
table are checked when the tables are accessed (for example, as part of aSELECT
orINSERT
statement). The checks ensure that the definitions of the tables and the parentMERGE
table definition match by comparing column order, types, sizes and associated indexes. If there is a difference between the tables, an error is returned and the statement fails. Because these checks take place when the tables are opened, any changes to the definition of a single table, including column changes, column ordering, and engine alterations will cause the statement to fail.The order of indexes in the
MERGE
table and its underlying tables should be the same. If you useALTER TABLE
to add aUNIQUE
index to a table used in aMERGE
table, and then useALTER TABLE
to add a nonunique index on theMERGE
table, the index ordering is different for the tables if there was already a nonunique index in the underlying table. (This happens becauseALTER TABLE
putsUNIQUE
indexes before nonunique indexes to facilitate rapid detection of duplicate keys.) Consequently, queries on tables with such indexes may return unexpected results.If you encounter an error message similar to ERROR 1017 (HY000): Can't find file: '
tbl_name
.MRG' (errno: 2), it generally indicates that some of the underlying tables do not use theMyISAM
storage engine. Confirm that all of these tables areMyISAM
.The maximum number of rows in a
MERGE
table is 264 (~1.844E+19; the same as for aMyISAM
table). It is not possible to merge multipleMyISAM
tables into a singleMERGE
table that would have more than this number of rows.The
MERGE
storage engine does not supportINSERT DELAYED
statements.Use of underlying
MyISAM
tables of differing row formats with a parentMERGE
table is currently known to fail. See Bug #32364.You cannot change the union list of a nontemporary
MERGE
table whenLOCK TABLES
is in effect. The following does not work:CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...; LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE; ALTER TABLE m1 ... UNION=(t1,t2) ...;
However, you can do this with a temporary
MERGE
table.You cannot create a
MERGE
table withCREATE ... SELECT
, neither as a temporaryMERGE
table, nor as a nontemporaryMERGE
table. For example:CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...;
Attempts to do this result in an error:
tbl_name
is notBASE TABLE
.In some cases, differing
PACK_KEYS
table option values among theMERGE
and underlying tables cause unexpected results if the underlying tables containCHAR
orBINARY
columns. As a workaround, useALTER TABLE
to ensure that all involved tables have the samePACK_KEYS
value. (Bug #50646)
The FEDERATED
storage engine lets you access data
from a remote MySQL database without using replication or cluster
technology. Querying a local FEDERATED
table
automatically pulls the data from the remote (federated) tables. No
data is stored on the local tables.
To include the FEDERATED
storage engine if you
build MySQL from source, invoke CMake with the
-DWITH_FEDERATED_STORAGE_ENGINE
option.
The FEDERATED
storage engine is not enabled by
default in the running server; to enable
FEDERATED
, you must start the MySQL server binary
using the --federated
option.
To examine the source for the FEDERATED
engine,
look in the storage/federated
directory of a
MySQL source distribution.
When you create a table using one of the standard storage engines
(such as MyISAM
, CSV
or
InnoDB
), the table consists of the table
definition and the associated data. When you create a
FEDERATED
table, the table definition is the
same, but the physical storage of the data is handled on a remote
server.
A FEDERATED
table consists of two elements:
A remote server with a database table, which in turn consists of the table definition (stored in the
.frm
file) and the associated table. The table type of the remote table may be any type supported by the remotemysqld
server, includingMyISAM
orInnoDB
.A local server with a database table, where the table definition matches that of the corresponding table on the remote server. The table definition is stored within the
.frm
file. However, there is no data file on the local server. Instead, the table definition includes a connection string that points to the remote table.
When executing queries and statements on a
FEDERATED
table on the local server, the
operations that would normally insert, update or delete
information from a local data file are instead sent to the remote
server for execution, where they update the data file on the
remote server or return matching rows from the remote server.
The basic structure of a FEDERATED
table setup
is shown in Figure 13.2, “FEDERATED
Table Structure”.
When a client issues an SQL statement that refers to a
FEDERATED
table, the flow of information
between the local server (where the SQL statement is executed) and
the remote server (where the data is physically stored) is as
follows:
The storage engine looks through each column that the
FEDERATED
table has and constructs an appropriate SQL statement that refers to the remote table.The statement is sent to the remote server using the MySQL client API.
The remote server processes the statement and the local server retrieves any result that the statement produces (an affected-rows count or a result set).
If the statement produces a result set, each column is converted to internal storage engine format that the
FEDERATED
engine expects and can use to display the result to the client that issued the original statement.
The local server communicates with the remote server using MySQL
client C API functions. It invokes
mysql_real_query()
to send the
statement. To read a result set, it uses
mysql_store_result()
and fetches
rows one at a time using
mysql_fetch_row()
.
To create a FEDERATED
table you should follow
these steps:
Create the table on the remote server. Alternatively, make a note of the table definition of an existing table, perhaps using the
SHOW CREATE TABLE
statement.Create the table on the local server with an identical table definition, but adding the connection information that links the local table to the remote table.
For example, you could create the following table on the remote server:
CREATE TABLE test_table ( id INT(20) NOT NULL AUTO_INCREMENT, name VARCHAR(32) NOT NULL DEFAULT '', other INT(20) NOT NULL DEFAULT '0', PRIMARY KEY (id), INDEX name (name), INDEX other_key (other) ) ENGINE=MyISAM DEFAULT CHARSET=latin1;
To create the local table that will be federated to the remote
table, there are two options available. You can either create the
local table and specify the connection string (containing the
server name, login, password) to be used to connect to the remote
table using the CONNECTION
, or you can use an
existing connection that you have previously created using the
CREATE SERVER
statement.
When you create the local table it must have an identical field definition to the remote table.
You can improve the performance of a
FEDERATED
table by adding indexes to the
table on the host. The optimization will occur because the query
sent to the remote server will include the contents of the
WHERE
clause and will be sent to the remote
server and subsequently executed locally. This reduces the
network traffic that would otherwise request the entire table
from the server for local processing.
To use the first method, you must specify the
CONNECTION
string after the engine type in a
CREATE TABLE
statement. For
example:
CREATE TABLE federated_table ( id INT(20) NOT NULL AUTO_INCREMENT, name VARCHAR(32) NOT NULL DEFAULT '', other INT(20) NOT NULL DEFAULT '0', PRIMARY KEY (id), INDEX name (name), INDEX other_key (other) ) ENGINE=FEDERATED DEFAULT CHARSET=latin1 CONNECTION='mysql://fed_user@remote_host:9306/federated/test_table';
CONNECTION
replaces the
COMMENT
used in some previous versions of
MySQL.
The CONNECTION
string contains the
information required to connect to the remote server containing
the table that will be used to physically store the data. The
connection string specifies the server name, login credentials,
port number and database/table information. In the example, the
remote table is on the server remote_host
,
using port 9306. The name and port number should match the host
name (or IP address) and port number of the remote MySQL server
instance you want to use as your remote table.
The format of the connection string is as follows:
scheme
://user_name
[:password
]@host_name
[:port_num
]/db_name
/tbl_name
Where:
scheme
: A recognized connection protocol. Onlymysql
is supported as thescheme
value at this point.user_name
: The user name for the connection. This user must have been created on the remote server, and must have suitable privileges to perform the required actions (SELECT
,INSERT
,UPDATE
, and so forth) on the remote table.password
: (Optional) The corresponding password foruser_name
.host_name
: The host name or IP address of the remote server.port_num
: (Optional) The port number for the remote server. The default is 3306.db_name
: The name of the database holding the remote table.tbl_name
: The name of the remote table. The name of the local and the remote table do not have to match.
Sample connection strings:
CONNECTION='mysql://username:password@hostname:port/database/tablename' CONNECTION='mysql://username@hostname/database/tablename' CONNECTION='mysql://username:password@hostname/database/tablename'
If you are creating a number of FEDERATED
tables on the same server, or if you want to simplify the
process of creating FEDERATED
tables, you can
use the CREATE SERVER
statement
to define the server connection parameters, just as you would
with the CONNECTION
string.
The format of the CREATE SERVER
statement is:
CREATE SERVERserver_name
FOREIGN DATA WRAPPERwrapper_name
OPTIONS (option
[,option
] ...)
The server_name
is used in the
connection string when creating a new
FEDERATED
table.
For example, to create a server connection identical to the
CONNECTION
string:
CONNECTION='mysql://fed_user@remote_host:9306/federated/test_table';
You would use the following statement:
CREATE SERVER fedlink FOREIGN DATA WRAPPER mysql OPTIONS (USER 'fed_user', HOST 'remote_host', PORT 9306, DATABASE 'federated');
To create a FEDERATED
table that uses this
connection, you still use the CONNECTION
keyword, but specify the name you used in the
CREATE SERVER
statement.
CREATE TABLE test_table ( id INT(20) NOT NULL AUTO_INCREMENT, name VARCHAR(32) NOT NULL DEFAULT '', other INT(20) NOT NULL DEFAULT '0', PRIMARY KEY (id), INDEX name (name), INDEX other_key (other) ) ENGINE=FEDERATED DEFAULT CHARSET=latin1 CONNECTION='fedlink/test_table';
The connection name in this example contains the name of the
connection (fedlink
) and the name of the
table (test_table
) to link to, separated by a
slash. If you specify only the connection name without a table
name, the table name of the local table is used instead.
For more information on CREATE
SERVER
, see Section 12.1.16, “CREATE SERVER
Синтаксис”.
The CREATE SERVER
statement
accepts the same arguments as the CONNECTION
string. The CREATE SERVER
statement updates the rows in the
mysql.servers
table. See the following table
for information on the correspondence between parameters in a
connection string, options in the CREATE
SERVER
statement, and the columns in the
mysql.servers
table. For reference, the
format of the CONNECTION
string is as
follows:
scheme
://user_name
[:password
]@host_name
[:port_num
]/db_name
/tbl_name
Описание | CONNECTION string | CREATE SERVER option | mysql.servers column |
---|---|---|---|
Connection scheme | scheme | wrapper_name | Wrapper |
Remote user | user_name | USER | Username |
Remote password | password | PASSWORD | Password |
Remote host | host_name | HOST | Host |
Remote port | port_num | PORT | Port |
Remote database | db_name | DATABASE | Db |
You should be aware of the following points when using the
FEDERATED
storage engine:
FEDERATED
tables may be replicated to other slaves, but you must ensure that the slave servers are able to use the user/password combination that is defined in theCONNECTION
string (or the row in themysql.servers
table) to connect to the remote server.
The following items indicate features that the
FEDERATED
storage engine does and does not
support:
The remote server must be a MySQL server.
The remote table that a
FEDERATED
table points to must exist before you try to access the table through theFEDERATED
table.It is possible for one
FEDERATED
table to point to another, but you must be careful not to create a loop.A
FEDERATED
table does not support indexes per se. Because access to the table is handled remotely, it is the remote table that supports the indexes. Care should be taken when creating aFEDERATED
table since the index definition from an equivalentMyISAM
or other table may not be supported. For example, creating aFEDERATED
table with an index prefix onVARCHAR
,TEXT
orBLOB
columns will fail. The following definition inMyISAM
is valid:CREATE TABLE `T1`(`A` VARCHAR(100),UNIQUE KEY(`A`(30))) ENGINE=MYISAM;
The key prefix in this example is incompatible with the
FEDERATED
engine, and the equivalent statement will fail:CREATE TABLE `T1`(`A` VARCHAR(100),UNIQUE KEY(`A`(30))) ENGINE=FEDERATED CONNECTION='MYSQL://127.0.0.1:3306/TEST/T1';
If possible, you should try to separate the column and index definition when creating tables on both the remote server and the local server to avoid these index issues.
Internally, the implementation uses
SELECT
,INSERT
,UPDATE
, andDELETE
, but notHANDLER
.The
FEDERATED
storage engine supportsSELECT
,INSERT
,UPDATE
,DELETE
,TRUNCATE TABLE
, and indexes. It does not supportALTER TABLE
, or any Data Definition Language statements that directly affect the structure of the table, other thanDROP TABLE
. The current implementation does not use prepared statements.FEDERATED
acceptsINSERT ... ON DUPLICATE KEY UPDATE
statements, but if a duplicate-key violation occurs, the statement fails with an error.Performance on a
FEDERATED
table when performing bulk inserts (for example, on aINSERT INTO ... SELECT ...
statement) is slower than with other table types because each selected row is treated as an individualINSERT
statement on theFEDERATED
table.Transactions are not supported.
FEDERATED
performs bulk-insert handling such that multiple rows are sent to the remote table in a batch. This provides a performance improvement and enables the remote table to perform improvement. Also, if the remote table is transactional, it enables the remote storage engine to perform statement rollback properly should an error occur. This capability has the following limitations:The size of the insert cannot exceed the maximum packet size between servers. If the insert exceeds this size, it is broken into multiple packets and the rollback problem can occur.
Bulk-insert handling does not occur for
INSERT ... ON DUPLICATE KEY UPDATE
.
There is no way for the
FEDERATED
engine to know if the remote table has changed. The reason for this is that this table must work like a data file that would never be written to by anything other than the database system. The integrity of the data in the local table could be breached if there was any change to the remote database.When using a
CONNECTION
string, you cannot use an '@' character in the password. You can get round this limitation by using theCREATE SERVER
statement to create a server connection.The
insert_id
andtimestamp
options are not propagated to the data provider.Any
DROP TABLE
statement issued against aFEDERATED
table drops only the local table, not the remote table.FEDERATED
tables do not work with the query cache.User-defined partitioning is not supported for
FEDERATED
tables.
The following additional resources are available for the
FEDERATED
storage engine:
A forum dedicated to the
FEDERATED
storage engine is available at http://forums.mysql.com/list.php?105.
The EXAMPLE
storage engine is a stub engine that
does nothing. Its purpose is to serve as an example in the MySQL
source code that illustrates how to begin writing new storage
engines. As such, it is primarily of interest to developers.
To enable the EXAMPLE
storage engine if you build
MySQL from source, invoke CMake with the
-DWITH_EXAMPLE_STORAGE_ENGINE
option.
To examine the source for the EXAMPLE
engine,
look in the storage/example
directory of a
MySQL source distribution.
When you create an EXAMPLE
table, the server
creates a table format file in the database directory. The file
begins with the table name and has an .frm
extension. No other files are created. No data can be stored into
the table. Retrievals return an empty result.
mysql>CREATE TABLE test (i INT) ENGINE = EXAMPLE;
Query OK, 0 rows affected (0.78 sec) mysql>INSERT INTO test VALUES(1),(2),(3);
ERROR 1031 (HY000): Table storage engine for 'test' doesn't » have this option mysql>SELECT * FROM test;
Empty set (0.31 sec)
The EXAMPLE
storage engine does not support
indexing.
Other storage engines may be available from third parties and community members that have used the Custom Storage Engine interface.
You can find more information on the list of third party storage engines on the MySQL Forge Storage Engines page.
Third party engines are not supported by MySQL. For further information, documentation, installation guides, bug reporting or for any help or assistance with these engines, please contact the developer of the engine directly.
Third party engines that are known to be available include the following; please see the MySQL Forge links provided for more information:
PrimeBase XT (PBXT): PBXT has been designed for modern, web-based, high concurrency environments.
RitmarkFS: RitmarkFS enables you to access and manipulate the file system using SQL queries. RitmarkFS also supports file system replication and directory change tracking.
Distributed Data Engine: The Distributed Data Engine is an Open Source project that is dedicated to provide a Storage Engine for distributed data according to workload statistics.
mdbtools: A pluggable storage engine that enables read-only access to Microsoft Access
.mdb
database files.solidDB for MySQL: solidDB Storage Engine for MySQL is an open source, transactional storage engine for MySQL Server. It is designed for mission-critical implementations that require a robust, transactional database. solidDB Storage Engine for MySQL is a multi-threaded storage engine that supports full ACID compliance with all expected transaction isolation levels, row-level locking, and Multi-Version Concurrency Control (MVCC) with nonblocking reads and writes.
BLOB Streaming Engine (MyBS): The Scalable BLOB Streaming infrastructure for MySQL will transform MySQL into a scalable media server capable of streaming pictures, films, MP3 files and other binary and text objects (BLOBs) directly in and out of the database.
For more information on developing a customer storage engine that can be used with the Pluggable Storage Engine Architecture, see Writing a Custom Storage Engine on MySQL Forge.