Содержание
- 16.1. MySQL Cluster Overview
- 16.1.1. MySQL Cluster Core Concepts
- 16.1.2. MySQL Cluster Nodes, Node Groups, Replicas, and Partitions
- 16.1.3. MySQL Cluster Hardware, Software, and Networking Requirements
- 16.1.4. MySQL Cluster Development History
- 16.1.5. MySQL Server using
InnoDB
Compared with MySQL Cluster - 16.1.6. Known Limitations of MySQL Cluster
- 16.2. MySQL Cluster Installation
- 16.2.1. Installing MySQL Cluster on Linux
- 16.2.2. Installing MySQL Cluster on Windows
- 16.2.3. Initial Configuration of MySQL Cluster
- 16.2.4. Initial Startup of MySQL Cluster
- 16.2.5. MySQL Cluster Пример with Tables and Data
- 16.2.6. Safe Shutdown and Restart of MySQL Cluster
- 16.2.7. Upgrading and Downgrading MySQL Cluster NDB 7.2
- 16.3. MySQL Cluster Configuration
- 16.4. MySQL Cluster Programs
- 16.4.1. MySQL Server Usage for MySQL Cluster
- 16.4.2. ndbd — The MySQL Cluster Data Node Daemon
- 16.4.3. ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)
- 16.4.4. ndb_mgmd — The MySQL Cluster Management Server Daemon
- 16.4.5. ndb_mgm — The MySQL Cluster Management Client
- 16.4.6. ndb_config — Extract MySQL Cluster Configuration Information
- 16.4.7. ndb_cpcd — Automate Testing for NDB Development
- 16.4.8. ndb_delete_all — Delete All Rows from an NDB Table
- 16.4.9. ndb_desc — Describe NDB Tables
- 16.4.10. ndb_drop_index — Drop Index from an NDB Table
- 16.4.11. ndb_drop_table — Drop an NDB Table
- 16.4.12. ndb_error_reporter — NDB Error-Reporting Utility
- 16.4.13. ndb_print_backup_file — Print NDB Backup File Contents
- 16.4.14. ndb_print_schema_file — Print NDB Schema File Contents
- 16.4.15. ndb_print_sys_file — Print NDB System File Contents
- 16.4.16. ndbd_redo_log_reader — Check and Print Content of Cluster Redo Log
- 16.4.17. ndb_restore — Restore a MySQL Cluster Backup
- 16.4.18. ndb_select_all — Print Rows from an NDB Table
- 16.4.19. ndb_select_count — Print Row Counts for NDB Tables
- 16.4.20. ndb_show_tables — Display List of NDB Tables
- 16.4.21. ndb_size.pl — NDBCLUSTER Size Requirement Estimator
- 16.4.22. ndb_waiter — Wait for MySQL Cluster to Reach a Given Status
- 16.4.23. Options Common to MySQL Cluster Programs
- 16.5. Management of MySQL Cluster
- 16.5.1. Summary of MySQL Cluster Start Phases
- 16.5.2. Commands in the MySQL Cluster Management Client
- 16.5.3. Online Backup of MySQL Cluster
- 16.5.4. Performing a Rolling Restart of a MySQL Cluster
- 16.5.5. Event Reports Generated in MySQL Cluster
- 16.5.6. MySQL Cluster Log Messages
- 16.5.7. MySQL Cluster Single User Mode
- 16.5.8. Quick Reference: MySQL Cluster SQL Statements
- 16.5.9. The
ndbinfo
MySQL Cluster Information Database - 16.5.10. MySQL Cluster Security Issues
- 16.5.11. MySQL Cluster Disk Data Tables
- 16.5.12. Adding MySQL Cluster Data Nodes Online
- 16.5.13. Distributed MySQL Privileges for MySQL Cluster
- 16.5.14. NDB API Statistics Counters and Variables
- 16.5.15. ndbmemcache
- 16.6. MySQL Cluster Replication
- 16.6.1. MySQL Cluster Replication: Abbreviations and Symbols
- 16.6.2. General Requirements for MySQL Cluster Replication
- 16.6.3. Known Issues in MySQL Cluster Replication
- 16.6.4. MySQL Cluster Replication Schema and Tables
- 16.6.5. Preparing the MySQL Cluster for Replication
- 16.6.6. Starting MySQL Cluster Replication (Single Replication Channel)
- 16.6.7. Using Two Replication Channels for MySQL Cluster Replication
- 16.6.8. Implementing Failover with MySQL Cluster Replication
- 16.6.9. MySQL Cluster Backups With MySQL Cluster Replication
- 16.6.10. MySQL Cluster Replication: Multi-Master and Circular Replication
- 16.6.11. MySQL Cluster Replication Conflict Resolution
- 16.7. Changes in MySQL Cluster
This chapter contains information about MySQL
Cluster, which is a high-availability, high-redundancy
version of MySQL adapted for the distributed computing environment.
Recent releases of MySQL Cluster use version 7 of the
NDBCLUSTER
storage engine (also known
as NDB
) to enable running several
computers with MySQL servers and other software in a cluster; the
latest releases available for production use incorporate
NDB
version 7.2.
Support for the NDBCLUSTER
storage
engine is not included in the standard MySQL Server 5.5 binaries
built by Oracle. Instead, users of MySQL Cluster binaries from
Oracle should upgrade to the most recent binary release of MySQL
Cluster for supported platforms—these include RPMs that should
work with most Linux distributions. MySQL Cluster users who build
from source should use the sources provided for MySQL Cluster.
(Locations where the sources can be obtained are listed later in
this section.)
This chapter contains information about MySQL Cluster NDB 7.2 releases through 5.5.20-ndb-7.2.5. Currently, the MySQL Cluster NDB 7.2 release series is Generally Available (GA), as is MySQL Cluster NDB 7.1. MySQL Cluster NDB 7.0 and MySQL Cluster NDB 6.3 are previous GA release series; although they are still supported, we recommend that new deployments use MySQL Cluster NDB 7.2. For information about MySQL Cluster NDB 7.1, MySQL Cluster NDB 7.0, and previous versions of MySQL Cluster, see MySQL Cluster NDB 6.X/7.X, in the MySQL 5.1 Manual.
Supported Platforms.
MySQL Cluster is currently available and supported on a number of
platforms. For exact levels of support available for on specific
combinations of operating system versions, operating system
distributions, and hardware platforms, please refer to
http://www.mysql.com/support/supportedplatforms/cluster.html
.
Availability. MySQL Cluster binary and source packages are available for supported platforms from http://dev.mysql.com/downloads/cluster/.
MySQL Cluster release numbers.
MySQL Cluster follows a somewhat different release pattern from
the mainline MySQL Server 5.5 series of releases. In this
Manual and other MySQL documentation, we
identify these and later MySQL Cluster releases employing a
version number that begins with “NDB”. This version
number is that of the NDBCLUSTER
storage engine used in the release, and not of the MySQL server
version on which the MySQL Cluster release is based.
Version strings used in MySQL Cluster software. The version string displayed by MySQL Cluster programs uses this format:
mysql-mysql_server_version
-ndb-ndb_engine_version
mysql_server_version
represents the
version of the MySQL Server on which the MySQL Cluster release is
based. For all MySQL Cluster NDB 6.x and 7.x releases, this is
“5.1”. ndb_engine_version
is
the version of the NDB
storage engine
used by this release of the MySQL Cluster software. You can see this
format used in the mysql client, as shown here:
shell>mysql
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.1.61-ndb-7.1.20 Source distribution Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>SELECT VERSION()\G
*************************** 1. row *************************** VERSION(): 5.1.61-ndb-7.1.20 1 row in set (0.00 sec)
This version string is also displayed in the output of the
SHOW
command in the ndb_mgm
client:
ndb_mgm> SHOW
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=1 @10.0.10.6 (5.5.20-ndb-7.2.5, Nodegroup: 0, Master)
id=2 @10.0.10.8 (5.5.20-ndb-7.2.5, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=3 @10.0.10.2 (5.5.20-ndb-7.2.5)
[mysqld(API)] 2 node(s)
id=4 @10.0.10.10 (5.5.20-ndb-7.2.5)
id=5 (not connected, accepting connect from any host)
The version string identifies the mainline MySQL version from which
the MySQL Cluster release was branched and the version of the
NDBCLUSTER
storage engine used. For
example, the full version string for MySQL Cluster NDB 7.2.4 (the
first MySQL Cluster production release based on MySQL Server 5.5) is
mysql-5.5.19-ndb-7.2.4
. From this we can
determine the following:
Since the portion of the version string preceding “
-ndb-
” is the base MySQL Server version, this means that MySQL Cluster NDB 7.2.4 derives from the MySQL 5.5.19, and contains all feature enhancements and bugfixes from MySQL 5.5 up to and including MySQL 5.5.19.Since the portion of the version string following “
-ndb-
” represents the version number of theNDB
(orNDBCLUSTER
) storage engine, MySQL Cluster NDB 7.2.4 uses version 7.2.4 of theNDBCLUSTER
storage engine.
New MySQL Cluster releases are numbered according to updates in the
NDB
storage engine, and do not necessarily
correspond in a one-to-one fashion with mainline MySQL Server
releases. For example, MySQL Cluster NDB 7.2.4 (as previously noted)
is based on MySQL 5.5.19, while MySQL Cluster NDB 7.2.0 was based on
MySQL 5.1.51 (version string:
mysql-5.1.51-ndb-7.2.0
).
Compatibility with standard MySQL 5.5 releases.
While many standard MySQL schemas and applications can work using
MySQL Cluster, it is also true that unmodified applications and
database schemas may be slightly incompatible or have suboptimal
performance when run using MySQL Cluster (see
Section 16.1.6, “Known Limitations of MySQL Cluster”). Most of these issues
can be overcome, but this also means that you are very unlikely to
be able to switch an existing application datastore—that
currently uses, for example, MyISAM
or InnoDB
—to use the
NDB
storage engine without allowing
for the possibility of changes in schemas, queries, and
applications. In addition, the MySQL Server and MySQL Cluster
codebases diverge considerably, so that the standard
mysqld cannot function as a drop-in replacement
for the version of mysqld supplied with MySQL
Cluster.
MySQL Cluster development source trees. MySQL Cluster development trees can also be accessed from https://code.launchpad.net/~mysql/:
The MySQL Cluster development sources maintained at https://code.launchpad.net/~mysql/ are licensed under the GPL. For information about obtaining MySQL sources using Bazaar and building them yourself, see Section 2.9.3, “Installing MySQL from a Development Source Tree”.
As with MySQL Server 5.5, MySQL Cluster NDB 7.2 is built using CMake.
Currently, MySQL Cluster NDB 7.0, MySQL Cluster NDB 7.1, and MySQL Cluster NDB 7.2 releases are all Generally Available (GA), although we recommend that new deployments use MySQL Cluster NDB 7.2. MySQL Cluster NDB 6.1, MySQL Cluster NDB 6.2, and MySQL Cluster NDB 6.3, are no longer in active development. For an overview of major features added in MySQL Cluster NDB 7.2, see Section 16.1.4, “MySQL Cluster Development History”. For an overview of major features added in past MySQL Cluster releases through MySQL Cluster NDB 7.1, see MySQL Cluster Development History.
This chapter represents a work in progress, and its contents are
subject to revision as MySQL Cluster continues to evolve. Additional
information regarding MySQL Cluster can be found on the MySQL Web
site at http://www.mysql.com/products/cluster/
.
Additional Resources. More information may be found in the following places:
For answers to some commonly asked questions about MySQL Cluster, see Section B.10, “MySQL FAQ: MySQL 5.5 and MySQL Cluster”.
The MySQL Cluster mailing list: http://lists.mysql.com/cluster.
The MySQL Cluster Forum: http://forums.mysql.com/list.php?25.
Many MySQL Cluster users and developers blog about their experiences with MySQL Cluster, and make feeds of these available through PlanetMySQL.
- 16.1.1. MySQL Cluster Core Concepts
- 16.1.2. MySQL Cluster Nodes, Node Groups, Replicas, and Partitions
- 16.1.3. MySQL Cluster Hardware, Software, and Networking Requirements
- 16.1.4. MySQL Cluster Development History
- 16.1.5. MySQL Server using
InnoDB
Compared with MySQL Cluster - 16.1.6. Known Limitations of MySQL Cluster
MySQL Cluster is a technology that enables clustering of in-memory databases in a shared-nothing system. The shared-nothing architecture enables the system to work with very inexpensive hardware, and with a minimum of specific requirements for hardware or software.
MySQL Cluster is designed not to have any single point of failure. In a shared-nothing system, each component is expected to have its own memory and disk, and the use of shared storage mechanisms such as network shares, network file systems, and SANs is not recommended or supported.
MySQL Cluster integrates the standard MySQL server with an in-memory
clustered storage engine called NDB
(which stands for “Network
DataBase”). In our
documentation, the term NDB
refers to
the part of the setup that is specific to the storage engine,
whereas “MySQL Cluster” refers to the combination of
one or more MySQL servers with the NDB
storage engine.
A MySQL Cluster consists of a set of computers, known as hosts, each running one or more processes. These processes, known as nodes, may include MySQL servers (for access to NDB data), data nodes (for storage of the data), one or more management servers, and possibly other specialized data access programs. The relationship of these components in a MySQL Cluster is shown here:

All these programs work together to form a MySQL Cluster (see
Section 16.4, “MySQL Cluster Programs”. When data is stored by the
NDB
storage engine, the tables (and
table data) are stored in the data nodes. Such tables are directly
accessible from all other MySQL servers (SQL nodes) in the cluster.
Thus, in a payroll application storing data in a cluster, if one
application updates the salary of an employee, all other MySQL
servers that query this data can see this change immediately.
Although a MySQL Cluster SQL node uses the mysqld server damon, it differs in a number of critical respects from the mysqld binary supplied with the MySQL 5.5 distributions, and the two versions of mysqld are not interchangeable.
In addition, a MySQL server that is not connected to a MySQL Cluster
cannot use the NDB
storage engine and
cannot access any MySQL Cluster data.
The data stored in the data nodes for MySQL Cluster can be mirrored; the cluster can handle failures of individual data nodes with no other impact than that a small number of transactions are aborted due to losing the transaction state. Because transactional applications are expected to handle transaction failure, this should not be a source of problems.
Individual nodes can be stopped and restarted, and can then rejoin the system (cluster). Rolling restarts (in which all nodes are restarted in turn) are used in making configuration changes and software upgrades (see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”). Rolling restarts are also used as part of the process of adding new data nodes online (see Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”). For more information about data nodes, how they are organized in a MySQL Cluster, and how they handle and store MySQL Cluster data, see Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”.
Backing up and restoring MySQL Cluster databases can be done using
the NDB
-native functionality found in the MySQL
Cluster management client and the ndb_restore
program included in the MySQL Cluster distribution. For more
information, see Section 16.5.3, “Online Backup of MySQL Cluster”, and
Section 16.4.17, “ndb_restore — Restore a MySQL Cluster Backup”. You can also
use the standard MySQL functionality provided for this purpose in
mysqldump and the MySQL server. See
Section 4.5.4, “mysqldump — A Database Backup Program”, for more information.
MySQL Cluster nodes can use a number of different transport mechanisms for inter-node communications, including TCP/IP using standard 100 Mbps or faster Ethernet hardware. It is also possible to use the high-speed Scalable Coherent Interface (SCI) protocol with MySQL Cluster, although this is not required to use MySQL Cluster. SCI requires special hardware and software; see Section 16.3.5, “Using High-Speed Interconnects with MySQL Cluster”, for more about SCI and using it with MySQL Cluster.
NDBCLUSTER
(also known as NDB
) is an in-memory
storage engine offering high-availability and data-persistence
features.
The NDBCLUSTER
storage engine can be
configured with a range of failover and load-balancing options,
but it is easiest to start with the storage engine at the cluster
level. MySQL Cluster's NDB
storage
engine contains a complete set of data, dependent only on other
data within the cluster itself.
The “Cluster” portion of MySQL Cluster is configured independently of the MySQL servers. In a MySQL Cluster, each part of the cluster is considered to be a node.
In many contexts, the term “node” is used to indicate a computer, but when discussing MySQL Cluster it means a process. It is possible to run multiple nodes on a single computer; for a computer on which one or more cluster nodes are being run we use the term cluster host.
There are three types of cluster nodes, and in a minimal MySQL Cluster configuration, there will be at least three nodes, one of each of these types:
Management node: The role of this type of node is to manage the other nodes within the MySQL Cluster, performing such functions as providing configuration data, starting and stopping nodes, running backup, and so forth. Because this node type manages the configuration of the other nodes, a node of this type should be started first, before any other node. An MGM node is started with the command ndb_mgmd.
Data node: This type of node stores cluster data. There are as many data nodes as there are replicas, times the number of fragments (see Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”). For example, with two replicas, each having two fragments, you need four data nodes. One replica is sufficient for data storage, but provides no redundancy; therefore, it is recommended to have 2 (or more) replicas to provide redundancy, and thus high availability. A data node is started with the command ndbd (see Section 16.4.2, “ndbd — The MySQL Cluster Data Node Daemon”) or ndbmtd (see Section 16.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”).
MySQL Cluster tables are normally stored completely in memory rather than on disk (this is why we refer to MySQL Cluster as an in-memory database). However, some MySQL Cluster data can be stored on disk; see Section 16.5.11, “MySQL Cluster Disk Data Tables”, for more information.
SQL node: This is a node that accesses the cluster data. In the case of MySQL Cluster, an SQL node is a traditional MySQL server that uses the
NDBCLUSTER
storage engine. An SQL node is a mysqld process started with the--ndbcluster
and--ndb-connectstring
options, which are explained elsewhere in this chapter, possibly with additional MySQL server options as well.An SQL node is actually just a specialized type of API node, which designates any application which accesses MySQL Cluster data. Another example of an API node is the ndb_restore utility that is used to restore a cluster backup. It is possible to write such applications using the NDB API. For basic information about the NDB API, see Getting Started with the NDB API.
It is not realistic to expect to employ a three-node setup in a production environment. Such a configuration provides no redundancy; to benefit from MySQL Cluster's high-availability features, you must use multiple data and SQL nodes. The use of multiple management nodes is also highly recommended.
For a brief introduction to the relationships between nodes, node groups, replicas, and partitions in MySQL Cluster, see Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”.
Configuration of a cluster involves configuring each individual node in the cluster and setting up individual communication links between nodes. MySQL Cluster is currently designed with the intention that data nodes are homogeneous in terms of processor power, memory space, and bandwidth. In addition, to provide a single point of configuration, all configuration data for the cluster as a whole is located in one configuration file.
The management server manages the cluster configuration file and the cluster log. Each node in the cluster retrieves the configuration data from the management server, and so requires a way to determine where the management server resides. When interesting events occur in the data nodes, the nodes transfer information about these events to the management server, which then writes the information to the cluster log.
In addition, there can be any number of cluster client processes
or applications. These include standard MySQL clients,
NDB
-specific API programs, and management
clients. These are described in the next few paragraphs.
Standard MySQL clients. MySQL Cluster can be used with existing MySQL applications written in PHP, Perl, C, C++, Java, Python, Ruby, and so on. Such client applications send SQL statements to and receive responses from MySQL servers acting as MySQL Cluster SQL nodes in much the same way that they interact with standalone MySQL servers.
MySQL clients using a MySQL Cluster as a data source can be
modified to take advantage of the ability to connect with multiple
MySQL servers to achieve load balancing and failover. For example,
Java clients using Connector/J 5.0.6 and later can use
jdbc:mysql:loadbalance://
URLs (improved in
Connector/J 5.1.7) to achieve load balancing transparently; for
more information about using Connector/J with MySQL Cluster, see
Using Connector/J with MySQL Cluster.
NDB
client programs.
Client programs can be written that access MySQL Cluster data
directly from the NDBCLUSTER
storage engine,
bypassing any MySQL Servers that may connected to the cluster,
using the NDB API, a high-level C++ API.
Such applications may be useful for specialized purposes where
an SQL interface to the data is not needed. For more
information, see The NDB API.
Beginning with MySQL Cluster NDB 7.1,
NDB
-specific Java applications can also be
written for MySQL Cluster, using the MySQL Cluster
Connector for Java. This MySQL Cluster Connector
includes ClusterJ, a high-level database
API similar to object-relational mapping persistence frameworks
such as Hibernate and JPA that connect directly to
NDBCLUSTER
, and so does not require access to a
MySQL Server. Support is also provided in MySQL Cluster NDB 7.1
and later for ClusterJPA, an OpenJPA
implementation for MySQL Cluster that leverages the strengths of
ClusterJ and JDBC; ID lookups and other fast operations are
performed using ClusterJ (bypassing the MySQL Server), while more
complex queries that can benefit from MySQL's query optimizer
are sent through the MySQL Server, using JDBC. See
Java and MySQL Cluster, and
The ClusterJ API and Data Object Model, for more
information.
The Memcache API for MySQL Cluster, implemented as the loadable ndbmemcache storage engine for memcached version 1.6 and later, is available beginning with MySQL Cluster NDB 7.2.2. This API can be used to provide a persistent MySQL Cluster data store, accessed using the memcache protocol.
The standard memcached caching engine is included in the MySQL Cluster NDB 7.2 distribution (7.2.2 and later). Each memcached server has direct access to data stored in MySQL Cluster, but is also able to cache data locally and to serve (some) requests from this local cache.
For more information, see Section 16.5.15, “ndbmemcache”.
Management clients. These clients connect to the management server and provide commands for starting and stopping nodes gracefully, starting and stopping message tracing (debug versions only), showing node versions and status, starting and stopping backups, and so on. An example of this type of program is the ndb_mgm management client supplied with MySQL Cluster (see Section 16.4.5, “ndb_mgm — The MySQL Cluster Management Client”). Such applications can be written using the MGM API, a C-language API that communicates directly with one or more MySQL Cluster management servers. For more information, see The MGM API.
Oracle also makes available MySQL Cluster Manager, which provides an advanced command-line interface simplifying many complex MySQL Cluster management tasks, such restarting a MySQL Cluster with a large number of nodes. The MySQL Cluster Manager client also supports commands for getting and setting the values of most node configuration parameters as well as mysqld server options and variables relating to MySQL Cluster. MySQL Cluster Manager 1.1 provides support for adding data nodes online. See the MySQL Cluster Manager User Manual, for more information.
Event logs. MySQL Cluster logs events by category (startup, shutdown, errors, checkpoints, and so on), priority, and severity. A complete listing of all reportable events may be found in Section 16.5.5, “Event Reports Generated in MySQL Cluster”. Event logs are of the two types listed here:
Cluster log: Keeps a record of all desired reportable events for the cluster as a whole.
Node log: A separate log which is also kept for each individual node.
Under normal circumstances, it is necessary and sufficient to keep and examine only the cluster log. The node logs need be consulted only for application development and debugging purposes.
Checkpoint.
Generally speaking, when data is saved to disk, it is said that
a checkpoint has been reached. More
specific to MySQL Cluster, a checkpoint is a point in time where
all committed transactions are stored on disk. With regard to
the NDB
storage engine, there are
two types of checkpoints which work together to ensure that a
consistent view of the cluster's data is maintained. These are
shown in the following list:
Local Checkpoint (LCP): This is a checkpoint that is specific to a single node; however, LCP's take place for all nodes in the cluster more or less concurrently. An LCP involves saving all of a node's data to disk, and so usually occurs every few minutes. The precise interval varies, and depends upon the amount of data stored by the node, the level of cluster activity, and other factors.
Global Checkpoint (GCP): A GCP occurs every few seconds, when transactions for all nodes are synchronized and the redo-log is flushed to disk.
This section discusses the manner in which MySQL Cluster divides and duplicates data for storage.
A number of concepts central to an understanding of this topic are discussed in the next few paragraphs.
(Data) Node. An ndbd process, which stores a replica —that is, a copy of the partition (see below) assigned to the node group of which the node is a member.
Each data node should be located on a separate computer. While it is also possible to host multiple ndbd processes on a single computer, such a configuration is not supported.
It is common for the terms “node” and “data node” to be used interchangeably when referring to an ndbd process; where mentioned, management nodes (ndb_mgmd processes) and SQL nodes (mysqld processes) are specified as such in this discussion.
Node Group. A node group consists of one or more nodes, and stores partitions, or sets of replicas (see next item).
The number of node groups in a MySQL Cluster is not directly
configurable; it is a function of the number of data nodes and of
the number of replicas
(NoOfReplicas
configuration parameter), as shown here:
[number_of_node_groups
] =number_of_data_nodes
/NoOfReplicas
Thus, a MySQL Cluster with 4 data nodes has 4 node groups if
NoOfReplicas
is set to 1
in the config.ini
file, 2 node groups if
NoOfReplicas
is set to 2,
and 1 node group if
NoOfReplicas
is set to 4.
Replicas are discussed later in this section; for more information
about NoOfReplicas
, see
Section 16.3.2.6, “Defining MySQL Cluster Data Nodes”.
All node groups in a MySQL Cluster must have the same number of data nodes.
You can add new node groups (and thus new data nodes) online, to a running MySQL Cluster; see Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”, for more information.
Partition. This is a portion of the data stored by the cluster. There are as many cluster partitions as nodes participating in the cluster. Each node is responsible for keeping at least one copy of any partitions assigned to it (that is, at least one replica) available to the cluster.
A replica belongs entirely to a single node; a node can (and usually does) store several replicas.
NDB
and user-defined partitioning.
MySQL Cluster normally partitions
NDBCLUSTER
tables automatically.
However, in MySQL 5.1 and later MySQL Cluster releases, it is
possible to employ user-defined partitioning with
NDBCLUSTER
tables. This is subject
to the following limitations:
Only
KEY
andLINEAR KEY
partitioning schemes can be used withNDBCLUSTER
tables.When using ndbd, the maximum number of partitions that may be defined explicitly for any
NDBCLUSTER
table is8 * [
. (The number of node groups in a MySQL Cluster is determined as discussed previously in this section.)number of node groups
]When using ndbmtd, this maximum is also affected by the number of local query handler threads, which is determined by the value of the
MaxNoOfExecutionThreads
configuration parameter. In such cases, the maxmimum number of partitions that may be defined explicitly for anNDB
table is equal to4 * MaxNoOfExecutionThreads * [
.number of node groups
]See Section 16.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
For more information relating to MySQL Cluster and user-defined partitioning, see Section 16.1.6, “Known Limitations of MySQL Cluster”, and Section 17.5.2, “Partitioning Limitations Relating to Storage Engines”.
Replica. This is a copy of a cluster partition. Each node in a node group stores a replica. Also sometimes known as a partition replica. The number of replicas is equal to the number of nodes per node group.
The following diagram illustrates a MySQL Cluster with four data nodes, arranged in two node groups of two nodes each; nodes 1 and 2 belong to node group 0, and nodes 3 and 4 belong to node group 1. Note that only data (ndbd) nodes are shown here; although a working cluster requires an ndb_mgm process for cluster management and at least one SQL node to access the data stored by the cluster, these have been omitted in the figure for clarity.

The data stored by the cluster is divided into four partitions, numbered 0, 1, 2, and 3. Each partition is stored—in multiple copies—on the same node group. Partitions are stored on alternate node groups as follows:
Partition 0 is stored on node group 0; a primary replica (primary copy) is stored on node 1, and a backup replica (backup copy of the partition) is stored on node 2.
Partition 1 is stored on the other node group (node group 1); this partition's primary replica is on node 3, and its backup replica is on node 4.
Partition 2 is stored on node group 0. However, the placing of its two replicas is reversed from that of Partition 0; for Partition 2, the primary replica is stored on node 2, and the backup on node 1.
Partition 3 is stored on node group 1, and the placement of its two replicas are reversed from those of partition 1. That is, its primary replica is located on node 4, with the backup on node 3.
What this means regarding the continued operation of a MySQL Cluster is this: so long as each node group participating in the cluster has at least one node operating, the cluster has a complete copy of all data and remains viable. This is illustrated in the next diagram.

In this example, where the cluster consists of two node groups of two nodes each, any combination of at least one node in node group 0 and at least one node in node group 1 is sufficient to keep the cluster “alive” (indicated by arrows in the diagram). However, if both nodes from either node group fail, the remaining two nodes are not sufficient (shown by the arrows marked out with an X); in either case, the cluster has lost an entire partition and so can no longer provide access to a complete set of all cluster data.
One of the strengths of MySQL Cluster is that it can be run on commodity hardware and has no unusual requirements in this regard, other than for large amounts of RAM, due to the fact that all live data storage is done in memory. (It is possible to reduce this requirement using Disk Data tables—see Section 16.5.11, “MySQL Cluster Disk Data Tables”, for more information about these.) Naturally, multiple and faster CPUs can enhance performance. Memory requirements for other MySQL Cluster processes are relatively small.
The software requirements for MySQL Cluster are also modest. Host operating systems do not require any unusual modules, services, applications, or configuration to support MySQL Cluster. For supported operating systems, a standard installation should be sufficient. The MySQL software requirements are simple: all that is needed is a production release of MySQL 5.1.61-ndb-7.0.31 or 5.1.61-ndb-7.1.20 to have MySQL Cluster support. It is not strictly necessary to compile MySQL yourself merely to be able to use MySQL Cluster. We assume that you are using the binaries appropriate to your platform, available from the MySQL Cluster software downloads page at http://dev.mysql.com/downloads/cluster/.
For communication between nodes, MySQL Cluster supports TCP/IP networking in any standard topology, and the minimum expected for each host is a standard 100 Mbps Ethernet card, plus a switch, hub, or router to provide network connectivity for the cluster as a whole. We strongly recommend that a MySQL Cluster be run on its own subnet which is not shared with machines not forming part of the cluster for the following reasons:
Security. Communications between MySQL Cluster nodes are not encrypted or shielded in any way. The only means of protecting transmissions within a MySQL Cluster is to run your MySQL Cluster on a protected network. If you intend to use MySQL Cluster for Web applications, the cluster should definitely reside behind your firewall and not in your network's De-Militarized Zone (DMZ) or elsewhere.
See Section 16.5.10.1, “MySQL Cluster Security and Networking Issues”, for more information.
Efficiency. Setting up a MySQL Cluster on a private or protected network enables the cluster to make exclusive use of bandwidth between cluster hosts. Using a separate switch for your MySQL Cluster not only helps protect against unauthorized access to MySQL Cluster data, it also ensures that MySQL Cluster nodes are shielded from interference caused by transmissions between other computers on the network. For enhanced reliability, you can use dual switches and dual cards to remove the network as a single point of failure; many device drivers support failover for such communication links.
Network communication and latency. MySQL Cluster requires communication between data nodes and API nodes (including SQL nodes), as well as between data nodes and other data nodes, to execute queries and updates. Communication latency between these processes can directly affect the observed performance and latency of user queries. In addition, to maintain consistency and service despite the silent failure of nodes, MySQL Cluster uses heartbeating and timeout mechanisms which treat an extended loss of communication from a node as node failure. This can lead to reduced redundancy. Recall that, to maintain data consistency, a MySQL Cluster shuts down when the last node in a node group fails. Thus, to avoid increasing the risk of a forced shutdown, breaks in communication between nodes should be avoided wherever possible.
The failure of a data or API node results in the abort of all uncommitted transactions involving the failed node. Data node recovery requires synchronization of the failed notde's data from a surviving data node, and re-establishment of disk-based redo and checkpoint logs, before the data node returns to service. This recovery can take some time, during which the Cluster operates with reduced redundancy.
Heartbeating relies on timely generation of heartbeat signals by all nodes. This may not be possible if the node is overloaded, has insufficient machine CPU due to sharing with other programs, or is experiencing delays due to swapping. If heartbeat generation is sufficiently delayed, other nodes treat the node that is slow to respond as failed.
This treatment of a slow node as a failed one may or may not be
desireable in some circumstances, depending on the impact of the
node's slowed operation on the rest of the cluster. When
setting timeout values such as
HeartbeatIntervalDbDb
and
HeartbeatIntervalDbApi
for
MySQL Cluster, care must be taken care to achieve quick detection,
failover, and return to service, while avoiding potentially
expensive false positives.
Where communication latencies between data nodes are expected to be higher than would be expected in a LAN environment (on the order of 100 µs), timeout parameters must be increased to ensure that any allowed periods of latency periods are well within configured timeouts. Increasing timeouts in this way has a corresponding effect on the worst-case time to detect failure and therefore time to service recovery.
LAN environments can typically be configured with stable low latency, and such that they can provide redundancy with fast failover. Individual link failures can be recovered from with minimal and controlled latency visible at the TCP level (where MySQL Cluster normally operates). WAN environments may offer a range of latencies, as well as redundancy with slower failover times. Individual link failures may require route changes to propagate before end-to-end connectivity is restored. At the TCP level this can appear as large latencies on individual channels. The worst-case observed TCP latency in these scenarios is related to the worst-case time for the IP layer to reroute around the failures.
SCI support. It is also possible to use the high-speed Scalable Coherent Interface (SCI) with MySQL Cluster, but this is not a requirement. See Section 16.3.5, “Using High-Speed Interconnects with MySQL Cluster”, for more about this protocol and its use with MySQL Cluster.
In this section, we discuss changes in the implementation of MySQL Cluster in MySQL MySQL Cluster NDB 7.2, as compared to MySQL Cluster NDB 7.1 and earlier releases. Changes and features most likely to be of interest are shown in the following table:
MySQL Cluster NDB 7.2 |
---|
MySQL Cluster NDB 7.2.1 and later MySQL Cluster NDB 7.2 releases are based on MySQL 5.5. For more information about new features in MySQL Server 5.5, see Section 1.4, “What Is New in MySQL 5.5”. |
Version 2 binary log row events, to provide support for improvements in
MySQL Cluster Replication conflict detection (see next
item). A given mysqld can be made to
use Version 1 or Version 2 binary logging row events with
the
--log-bin-use-v1-row-events
option. |
Two new “primary wins” conflict detection and resolution
functions
NDB$EPOCH()
and
NDB$EPOCH_TRANS()
for use in replication setups with 2 MySQL Clusters. For
more information, see
Section 16.6, “MySQL Cluster Replication”. |
Distribution of MySQL users and privileges across MySQL Cluster SQL nodes is now supported—see Section 16.5.13, “Distributed MySQL Privileges for MySQL Cluster”. |
Improved support for distributed pushed-down joins, which greatly improve performance for many joins that can be executed in parallel on the data nodes. |
Default values for a number of data node configuration parameters such
as
HeartbeatIntervalDbDb
and
ArbitrationTimeout
have been improved. |
Support for the Memcache API using the loadable ndbmemcache storage engine. See Section 16.5.15, “ndbmemcache”. |
This section contains information about MySQL Cluster NDB 7.2 releases through 5.5.20-ndb-7.2.5, which is currently available for use in production beginning with MySQL Cluster NDB 7.2.4. MySQL Cluster NDB 7.1, MySQL Cluster NDB 7.0, and MySQL Cluster NDB 6.3 are previous GA release series; although these are still supported, we recommend that new deployments use MySQL Cluster NDB 7.2. For information about MySQL Cluster NDB 7.1 and previous releases, see MySQL Cluster NDB 6.X/7.X, in the MySQL 5.1 Manual.
The following improvements to MySQL Cluster have been made in MySQL Cluster NDB 7.2.
Based on MySQL Server 5.5. Previous MySQL Cluster release series, including MySQL Cluster NDB 7.1, used MySQL 5.1 as a base. Beginning with MySQL Cluster NDB 7.2.1, MySQL Cluster NDB 7.2 is based on MySQL Server 5.5, so that MySQL Cluster users can benefit from MySQL 5.5's improvements in scalability and performance monitoring. As with MySQL 5.5, MySQL Cluster NDB 7.2.1 and later use CMake for configuring and building from source inh place of GNU autotools (used in MySQL 5.1 and MySQL Cluster releases based on MySQL 5.1). For more information about changes and improvements in MySQL 5.5, see Section 1.4, “What Is New in MySQL 5.5”.
Conflict detection using GCI Reflection. MySQL Cluster Replication implements a new “primary wins” conflict detection and resolution mechanism. GCI Reflection applies in two-cluster circulation “active-active” replication setups, tracking the order in which changes are applied on the MySQL Cluster designated as primary relative to changes originating on the other MySQL Cluster (referred to as the secondary). This relative ordering is used to determine whether changes originating on the slave are concurrent with any changes that originate locally, and are therefore potentially in conflict. Two new conflict detection functions are added: When using
NDB$EPOCH()
, rows that are out of sync on the secondary are realigned with those on the primary; withNDB$EPOCH_TRANS()
, this realignment is applied to transactions. For more information, see Section 16.6.11, “MySQL Cluster Replication Conflict Resolution”.Version 2 binary log row events. A new format for binary log row events, known as Version 2 binary log row events, provides support for improvements in MySQL Cluster Replication conflict detection (see previous item) and is intended to facilitate further improvements in MySQL Replication. You can cause a given mysqld use Version 1 or Version 2 binary logging row events with the
--log-bin-use-v1-row-events
option. For backwards compatiblity, Version 2 binary log row events are also available in MySQL Cluster NDB 7.0 (7.0.27 and later) and MySQL Cluster NDB 7.1 (7.1.16 and later). However, MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1 continue to use Version 1 binary log row events as the default, whereas the default in MySQL Cluster NDB 7.2.1 and later is use Version 2 row events for binary logging.Distribution of MySQL users and privileges. Automatic distribution of MySQL users and privileges across all SQL nodes in a given MySQL Cluster is now supported. To enable this support, you must first import an SQL script
share/mysql/ndb_dist_priv.sql
that is included with the MySQL Cluster NDB 7.2 distribution. This script creates several stored procedures which you can use to enable privilege distribution and perform related tasks.When a new MySQL Server joins a MySQL Cluster where privilege distribution is in effect, it also participates in the privilege distribution automatically.
Once privilege distribution is enabled, all changes to the grant tables made on any mysqld attached to the cluster are immediately available on any other attached MySQL Servers. This is true whether the changes are made using
CREATE USER
,GRANT
, or any of the other statements described elsewhere in this Manual (see Section 12.7.1, “Account Management Statements”.) This includes privileges relating to stored routines and views; however, automatic distribution of the views or stored routines themselves is not currently supported.For more information, see Section 16.5.13, “Distributed MySQL Privileges for MySQL Cluster”.
Distributed pushed-down joins. Many joins can now be pushed down to the NDB kernel for processing on MySQL Cluster data nodes. Previously, a join was handled in MySQL Cluster by means of repeated accesses of
NDB
by the SQL node; however, when pushed-down joins are enabled, a pushable join is sent in its entirety to the data nodes, where it can be distributed among the data nodes and executed in parallel on multiple copies of the data, with a single, merged result being returned to mysqld. This can reduce greatly the number of round trips between an SQL node and the data nodes required to handle such a join, leading to greatly improved performance of join processing.It is possible to determine when joins can be pushed down to the data nodes by examining the join with
EXPLAIN
. A number of new system status variables (Ndb_pushed_queries_defined
,Ndb_pushed_queries_dropped
,Ndb_pushed_queries_executed
, andNdb_pushed_reads
) and additions to thecounters
table (in thendbinfo
information database) can also be helpful in determining when and how well joins are being pushed down.More information and examples are available in the description of the
ndb_join_pushdown
server system variable. See also the description of the status variables referenced in the previous paragraph, as well as Section 16.5.9.3, “Thendbinfo counters
Table”.Improved default values for data node configuration parameters. In order to provide more resiliency to environmental issues and better handling of some potential failure scenarios, and to perform more reliably with increases in memory and other resource requirements brought about by recent improvements in join handling by
NDB
, the default values for a number of MySQL Cluster data node configuration parameters have been changed. The parameters and changes are described in the following list:HeartbeatIntervalDbDb
: Default increased from 1500 ms to 5000 ms.ArbitrationTimeout
: Default increased from 3000 ms to 7500 ms.TimeBetweenEpochsTimeout
: Now effectively disabled by default (default changed from 4000 ms to 0).SharedGlobalMemory
: Default increased from 20 MB to 128 MB.MaxParallelScansPerFragment
: Default increased from 32 to 256.
In addition, the value computed for
MaxNoOfLocalScans
when this parameter is not set inconfig.ini
has been increased by a factor of 4.Fail-fast data nodes. Beginning with MySQL Cluster NDB 7.2.1, data nodes handle corrupted tuples in a fail-fast manner by default. This is a change from previous versions of MySQL Cluster where this behavior had to be enabled explicitly by enabling the
CrashOnCorruptedTuple
configuration parameter. In MySQL Cluster NDB 7.2.1 and later, this parameter is enabled by default and must be explicitly disabled, in which case data nodes merely log a warning whenever they detect a corrupted tuple.Memcache API support (ndbmemcache). The Memcached server is a distributed in-memory caching server that uses a simple text-based protocol. It is often employed with key-value stores. The Memcache API for MySQL Cluster, available beginning with MySQL Cluster NDB 7.2.2, is implemented as a loadable storage engine for memcached version 1.6 and later. This API can be used to access a persistent MySQL Cluster data store employing the memcache protocol. It is also possible for the memcached server to provide a strictly defined interface to existing MySQL Cluster tables.
Each memcache server can both cache data locally and access data stored in MySQL Cluster directly. Cacheing policies are configurable. For more information, see Section 16.5.15, “ndbmemcache”.
MySQL Server offers a number of choices in storage engines. Since
both NDBCLUSTER
and
InnoDB
can serve as transactional
MySQL storage engines, users of MySQL Server sometimes become
interested in MySQL Cluster. They see
NDB
as a possible alternative or
upgrade to the default InnoDB
storage
engine in MySQL 5.5. While NDB
and
InnoDB
share common characteristics,
there are differences in architecture and implementation, so that
some existing MySQL Server applications and usage scenarios can be
a good fit for MySQL Cluster, but not all of them.
In this section, we discuss and compare some characteristics of
the NDB
storage engine used by MySQL
Cluster NDB 7.2 with InnoDB
used in
MySQL 5.5. The next few sections provide a technical comparison.
In many instances, decisions about when and where to use MySQL
Cluster must be made on a case-by-case basis, taking all factors
into consideration. While it is beyond the scope of this
documentation to provide specifics for every conceivable usage
scenario, we also attempt to offer some very general guidance on
the relative suitability of some common types of applications for
NDB
as opposed to
InnoDB
backends.
Recent MySQL Cluster NDB 7.2 releases use a
mysqld based on MySQL 5.5, including support
for InnoDB
1.1. While it is possible
to use InnoDB
tables with MySQL Cluster, such
tables are not clustered. It is also not possible to use programs
or libraries from a MySQL Cluster NDB 7.2 distribution with MySQL
Server 5.5, or the reverse.
While it is also true that some types of common business
applications can be run either on MySQL Cluster or on MySQL Server
(most likely using the InnoDB
storage
engine), there are some important architectural and implementation
differences. Section 16.1.5.1, “Differences Between the NDB
and
InnoDB
Storage Engines”,
provides a summary of the these differences. Due to the
differences, some usage scenarios are clearly more suitable for
one engine or the other; see
Section 16.1.5.2, “NDB
and InnoDB
Workloads”. This in turn
has an impact on the types of applications that better suited for
use with NDB
or
InnoDB
. See
Section 16.1.5.3, “NDB
and InnoDB
Feature Usage Summary”, for a comparison
of the relative suitability of each for use in common types of
database applications.
For information about the relative characteristics of the
NDB
and
MEMORY
storage engines, see
When to Use MEMORY
or MySQL Cluster.
See Глава 13, Storage Engines, for additional information about MySQL storage engines.
The MySQL Cluster NDB
storage
engine is implemented using a distributed, shared-nothing
architecture, which causes it to behave differently from
InnoDB
in a number of ways. For
those unaccustomed to working with
NDB
, unexpected behaviors can arise
due to its distributed nature with regard to transactions,
foreign keys, table limits, and other characteristics. These are
shown in the following table:
Feature |
|
MySQL Cluster |
---|---|---|
MySQL Server Version | 5.5 | 5.5 |
|
|
|
MySQL Cluster Version | N/A |
|
Storage Limits | 64TB | 3TB (Practical upper limit based on 48 data nodes with 64GB RAM each; can be increased with disk-based data and BLOBs) |
Foreign Keys | Yes | No
(Ignored, as with |
Transactions | All standard types | |
MVCC Non-Blocking Reads | Yes | No |
Data Compression | Yes | No (MySQL Cluster checkpoint and backup files can be compressed) |
Large Row Support (> 14K) |
Supported for (Using these types to store very large amounts of data can lower MySQL Cluster performance) | |
Replication Support | Asynchronous and semi-synchronous replication using MySQL Replication | Automatic synchronous replication within a MySQL Cluster. Asynchronous replication between MySQL Clusters, using MySQL Replication |
Scaleout for Read Operations | Yes (MySQL Replication) | Yes (Automatic partitioning in MySQL Cluster; MySQL Replication) |
Scaleout for Write Operations | Requires application-level partitioning (sharding) | Yes (Automatic partitioning in MySQL Cluster is transparent to applications) |
High Availability (HA) | Requires additional software | Yes (Designed for 99.999% uptime) |
Node Failure Recovery and Failover | Requires additional software | Automatic (Key element in MySQL Cluster architecture) |
Time for Node Failure Recovery | 30 seconds or longer | Typically < 1 second |
Real-Time Performance | No | Yes (Low latency) |
In-Memory Tables | No | Yes (Some data can optionally be stored on disk; both in-memory and disk data storage are durable) |
NoSQL Access to Storage Engine | Native memcached interface in development (see the MySQL Dev Zone article NoSQL to MySQL with Memcached) | Yes Multiple APIs, including Memcached, Java, JPA, C++, and HTTP/REST |
Concurrent and Parallel Writes | Not supported | Up to 48 writers, optimized for concurrent writes |
Conflict Detection and Resolution (Multiple Replication Masters) | No | Yes |
Hash Indexes | No | Yes |
Online Addition of Nodes | Read-only replicas using MySQL Replication | Yes (all node types) |
Online Upgrades | No | Yes |
Online Schema Modifications | No | Yes |
Real-Time Performance | No | Yes |
MySQL Cluster has a range of unique attributes that make it
ideal to serve applications requiring high availability, fast
failover, high throughput, and low latency. Due to its
distributed architecture and multi-node implementation, MySQL
Cluster also has specific constraints that may keep some
workloads from performing well. A number of major differences in
behavior between the NDB
and
InnoDB
storage engines with regard
to some common types of database-driven application workloads
are shown in the following table::
Workload |
MySQL Cluster ( | |
---|---|---|
High-Volume OLTP Applications | Yes | Yes |
DSS Applications (data marts, analytics) | Yes | Limited (Join operations across OLTP datasets not exceeding 3TB in size) |
Custom Applications | Yes | Yes |
Packaged Applications | Yes | Limited (should be mostly primary key access, without any requirement for foreign keys) |
In-Network Telecoms Applications (HLR, HSS, SDP) | No | Yes |
Session Management and Caching | Yes | Yes |
E-Commerce Applications | Yes | Yes |
User Profile Management, AAA Protocol | Yes | Yes |
When comparing application feature requirements to the
capabilities of InnoDB
with
NDB
, some are clearly more
compatible with one storage engine than the other. For example,
since NDB
does not support foreign
keys, an application that requires them and cannot be
re-engineered to remove this requirement is likely not to be a
good match for MySQL Cluster.
The following table shows required supported features for applications according to which of these two storage engines each of them is usually better suited:
Preferred application requirements for
|
Preferred application requirements for
|
---|---|
|
|
- 16.1.6.1. Noncompliance with SQL Синтаксис in MySQL Cluster
- 16.1.6.2. Limits and Differences of MySQL Cluster from Standard MySQL Limits
- 16.1.6.3. Limits Relating to Transaction Handling in MySQL Cluster
- 16.1.6.4. MySQL Cluster Error Handling
- 16.1.6.5. Limits Associated with Database Objects in MySQL Cluster
- 16.1.6.6. Unsupported or Missing Features in MySQL Cluster
- 16.1.6.7. Limitations Relating to Performance in MySQL Cluster
- 16.1.6.8. Issues Exclusive to MySQL Cluster
- 16.1.6.9. Limitations Relating to MySQL Cluster Disk Data Storage
- 16.1.6.10. Limitations Relating to Multiple MySQL Cluster Nodes
- 16.1.6.11. Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x
In the sections that follow, we discuss known limitations in
current releases of MySQL Cluster as compared with the features
available when using the MyISAM
and
InnoDB
storage engines. If you check the
“Cluster” category in the MySQL bugs database at
http://bugs.mysql.com, you can find known bugs in
the following categories under “MySQL Server:” in the
MySQL bugs database at http://bugs.mysql.com, which
we intend to correct in upcoming releases of MySQL Cluster:
MySQL Cluster
Cluster Direct API (NDBAPI)
Cluster Disk Data
Cluster Replication
ClusterJ
This information is intended to be complete with respect to the conditions just set forth. You can report any discrepancies that you encounter to the MySQL bugs database using the instructions given in Section 1.7, “How to Report Bugs or Problems”. If we do not plan to fix the problem in MySQL Cluster NDB 6.X or 7.X, we will add it to the list.
See Section 16.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x” for a list of issues in MySQL Cluster in MySQL 5.1 that have been resolved in the current version.
Limitations and other issues specific to MySQL Cluster Replication are described in Section 16.6.3, “Known Issues in MySQL Cluster Replication”.
Some SQL statements relating to certain MySQL features produce
errors when used with NDB
tables,
as described in the following list:
Temporary tables. Temporary tables are not supported. Trying either to create a temporary table that uses the
NDB
storage engine or to alter an existing temporary table to useNDB
fails with the error Table storage engine 'ndbcluster' does not support the create option 'TEMPORARY'.Indexes and keys in
NDB
tables. Keys and indexes on MySQL Cluster tables are subject to the following limitations:Column width. Attempting to create an index on an
NDB
table column whose width is greater than 3072 bytes succeeds, but only the first 3072 bytes are actually used for the index. In such cases, a warning Specified key was too long; max key length is 3072 bytes is issued, and aSHOW CREATE TABLE
statement shows the length of the index as 3072.TEXT
andBLOB
columns. You cannot create indexes onNDB
table columns that use any of theTEXT
orBLOB
data types.FULLTEXT
indexes. TheNDB
storage engine does not supportFULLTEXT
indexes, which are possible forMyISAM
tables only.However, you can create indexes on
VARCHAR
columns ofNDB
tables.USING HASH
keys andNULL
. Using nullable columns in unique keys and primary keys means that queries using these columns are handled as full table scans. To work around this issue, make the columnNOT NULL
, or re-create the index without theUSING HASH
option.Prefixes. There are no prefix indexes; only entire columns can be indexed. (The size of an
NDB
column index is always the same as the width of the column in bytes, up to and including 3072 bytes, as described earlier in this section. Also see Section 16.1.6.6, “Unsupported or Missing Features in MySQL Cluster”, for additional information.)BIT
columns. ABIT
column cannot be a primary key, unique key, or index, nor can it be part of a composite primary key, unique key, or index.AUTO_INCREMENT
columns. Like other MySQL storage engines, theNDB
storage engine can handle a maximum of oneAUTO_INCREMENT
column per table. However, in the case of a Cluster table with no explicit primary key, anAUTO_INCREMENT
column is automatically defined and used as a “hidden” primary key. For this reason, you cannot define a table that has an explicitAUTO_INCREMENT
column unless that column is also declared using thePRIMARY KEY
option. Attempting to create a table with anAUTO_INCREMENT
column that is not the table's primary key, and using theNDB
storage engine, fails with an error.
MySQL Cluster and geometry data types. Geometry data types (
WKT
andWKB
) are supported forNDB
tables. However, spatial indexes are not supported.Character sets and binary log files. Currently, the
ndb_apply_status
andndb_binlog_index
tables are created using thelatin1
(ASCII) character set. Because names of binary logs are recorded in this table, binary log files named using non-Latin characters are not referenced correctly in these tables. This is a known issue, which we are working to fix. (Bug #50226)To work around this problem, use only Latin-1 characters when naming binary log files or setting any the
--basedir
,--log-bin
, or--log-bin-index
options.Creating
NDBCLUSTER
tables with user-defined partitioning. Support for user-defined partitioning for MySQL Cluster in MySQL 5.1 (including MySQL Cluster NDB 6.X and 7.X through 7.1) is restricted to [LINEAR
]KEY
partitioning. Beginning with MySQL 5.1.12, using any other partitioning type withENGINE=NDB
orENGINE=NDBCLUSTER
in aCREATE TABLE
statement results in an error.Default partitioning scheme. As of MySQL 5.1.6, all MySQL Cluster tables are by default partitioned by
KEY
using the table's primary key as the partitioning key. If no primary key is explicitly set for the table, the “hidden” primary key automatically created by theNDBCLUSTER
storage engine is used instead. For additional discussion of these and related issues, see Section 17.2.5, “KEY
Partitioning”.CREATE TABLE
andALTER TABLE
statements that would cause a user-partitionedNDBCLUSTER
table not to meet either or both of the following two requirements are not permitted, and fail with an error:The table must have an explicit primary key.
All columns listed in the table's partitioning expression must be part of the primary key.
Exception. If a user-partitioned
NDBCLUSTER
table is created using an empty column-list (that is, usingPARTITION BY [LINEAR] KEY()
), then no explicit primary key is required.Maximum number of partitions for
NDBCLUSTER
tables. The maximum number of partitions that can defined for aNDBCLUSTER
table when employing user-defined partitioning is 8 per node group. (See Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”, for more information about MySQL Cluster node groups.DROP PARTITION
not supported. It is not possible to drop partitions fromNDB
tables usingALTER TABLE ... DROP PARTITION
. The other partitioning extensions toALTER TABLE
—ADD PARTITION
,REORGANIZE PARTITION
, andCOALESCE PARTITION
—are supported for Cluster tables, but use copying and so are not optimized. See Section 17.3.1, “Management ofRANGE
andLIST
Partitions” and Section 12.1.7, “ALTER TABLE
Синтаксис”.Row-based replication. When using row-based replication with MySQL Cluster, binary logging cannot be disabled. That is, the
NDB
storage engine ignores the value ofsql_log_bin
. (Bug #16680)
In this section, we list limits found in MySQL Cluster that either differ from limits found in, or that are not found in, standard MySQL.
Memory usage and recovery.
Memory consumed when data is inserted into an
NDB
table is not automatically
recovered when deleted, as it is with other storage engines.
Instead, the following rules hold true:
A
DELETE
statement on anNDB
table makes the memory formerly used by the deleted rows available for re-use by inserts on the same table only. However, this memory can be made available for general re-use by performingOPTIMIZE TABLE
.A rolling restart of the cluster also frees any memory used by deleted rows. See Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”.
A
DROP TABLE
orTRUNCATE TABLE
operation on anNDB
table frees the memory that was used by this table for re-use by anyNDB
table, either by the same table or by anotherNDB
table.ЗамечаниеRecall that
TRUNCATE TABLE
drops and re-creates the table. See Section 12.1.33, “TRUNCATE TABLE
Синтаксис”.Limits imposed by the cluster's configuration. A number of hard limits exist which are configurable, but available main memory in the cluster sets limits. See the complete list of configuration parameters in Section 16.3.2, “MySQL Cluster Configuration Files”. Most configuration parameters can be upgraded online. These hard limits include:
Database memory size and index memory size (
DataMemory
andIndexMemory
, respectively).DataMemory
is allocated as 32KB pages. As eachDataMemory
page is used, it is assigned to a specific table; once allocated, this memory cannot be freed except by dropping the table.See Section 16.3.2.6, “Defining MySQL Cluster Data Nodes”, for more information.
The maximum number of operations that can be performed per transaction is set using the configuration parameters
MaxNoOfConcurrentOperations
andMaxNoOfLocalOperations
.ЗамечаниеBulk loading,
TRUNCATE TABLE
, andALTER TABLE
are handled as special cases by running multiple transactions, and so are not subject to this limitation.Different limits related to tables and indexes. For example, the maximum number of ordered indexes in the cluster is determined by
MaxNoOfOrderedIndexes
, and the maximum number of ordered indexes per table is 16.
Node and data object maximums. The following limits apply to numbers of cluster nodes and metadata objects:
The maximum number of data nodes is 48.
A data node must have a node ID in the range of 1 to 49, inclusive. (Management and API nodes may use node IDs in the range 1 to 255, inclusive.)
The total maximum number of nodes in a MySQL Cluster is 255. This number includes all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers.
The maximum number of metadata objects in current versions of MySQL Cluster is 20320. This limit is hard-coded.
See Section 16.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.
A number of limitations exist in MySQL Cluster with regard to the handling of transactions. These include the following:
Transaction isolation level. The
NDBCLUSTER
storage engine supports only theREAD COMMITTED
transaction isolation level. (InnoDB
, for example, supportsREAD COMMITTED
,READ UNCOMMITTED
,REPEATABLE READ
, andSERIALIZABLE
.) See Section 16.5.3.4, “MySQL Cluster Backup Troubleshooting”, for information on how this can affect backing up and restoring Cluster databases.)Transactions and
BLOB
orTEXT
columns.NDBCLUSTER
stores only part of a column value that uses any of MySQL'sBLOB
orTEXT
data types in the table visible to MySQL; the remainder of theBLOB
orTEXT
is stored in a separate internal table that is not accessible to MySQL. This gives rise to two related issues of which you should be aware whenever executingSELECT
statements on tables that contain columns of these types:For any
SELECT
from a MySQL Cluster table: If theSELECT
includes aBLOB
orTEXT
column, theREAD COMMITTED
transaction isolation level is converted to a read with read lock. This is done to guarantee consistency.For any
SELECT
which uses a unique key lookup to retrieve any columns that use any of theBLOB
orTEXT
data types and that is executed within a transaction, a shared read lock is held on the table for the duration of the transaction—that is, until the transaction is either committed or aborted.This issue does not occur for queries that use index or table scans, even against
NDB
tables havingBLOB
orTEXT
columns.For example, consider the table
t
defined by the followingCREATE TABLE
statement:CREATE TABLE t ( a INT NOT NULL AUTO_INCREMENT PRIMARY KEY, b INT NOT NULL, c INT NOT NULL, d TEXT, INDEX i(b), UNIQUE KEY u(c) ) ENGINE = NDB,
Either of the following queries on
t
causes a shared read lock, because the first query uses a primary key lookup and the second uses a unique key lookup:SELECT * FROM t WHERE a = 1; SELECT * FROM t WHERE c = 1;
However, none of the four queries shown here causes a shared read lock:
SELECT * FROM t WHERE b 1; SELECT * FROM t WHERE d = '1'; SELECT * FROM t; SELECT b,c WHERE a = 1;
This is because, of these four queries, the first uses an index scan, the second and third use table scans, and the fourth, while using a primary key lookup, does not retrieve the value of any
BLOB
orTEXT
columns.You can help minimize issues with shared read locks by avoiding queries that use unique key lookups that retrieve
BLOB
orTEXT
columns, or, in cases where such queries are not avoidable, by committing transactions as soon as possible afterward.
Rollbacks. There are no partial transactions, and no partial rollbacks of transactions. A duplicate key or similar error causes the entire transaction to be rolled back.
This behavior differs from that of other transactional storage engines such as
InnoDB
that may roll back individual statements.Transactions and memory usage. As noted elsewhere in this chapter, MySQL Cluster does not handle large transactions well; it is better to perform a number of small transactions with a few operations each than to attempt a single large transaction containing a great many operations. Among other considerations, large transactions require very large amounts of memory. Because of this, the transactional behavior of a number of MySQL statements is effected as described in the following list:
TRUNCATE TABLE
is not transactional when used onNDB
tables. If aTRUNCATE TABLE
fails to empty the table, then it must be re-run until it is successful.DELETE FROM
(even with noWHERE
clause) is transactional. For tables containing a great many rows, you may find that performance is improved by using severalDELETE FROM ... LIMIT ...
statements to “chunk” the delete operation. If your objective is to empty the table, then you may wish to useTRUNCATE TABLE
instead.LOAD DATA
statements.LOAD DATA INFILE
is not transactional when used onNDB
tables.ImportantWhen executing a
LOAD DATA INFILE
statement, theNDB
engine performs commits at irregular intervals that enable better utilization of the communication network. It is not possible to know ahead of time when such commits take place.LOAD DATA FROM MASTER
is not supported in MySQL Cluster.ALTER TABLE
and transactions. When copying anNDB
table as part of anALTER TABLE
, the creation of the copy is nontransactional. (In any case, this operation is rolled back when the copy is deleted.)
Transactions and the
COUNT()
function. When using MySQL Cluster Replication, it is not possible to guarantee the transactional consistency of theCOUNT()
function on the slave. In other words, when performing on the master a series of statements (INSERT
,DELETE
, or both) that changes the number of rows in a table within a single transaction, executingSELECT COUNT(*) FROM
queries on the slave may yield intermediate results. This is due to the fact thattable
SELECT COUNT(...)
may perform dirty reads, and is not a bug in theNDB
storage engine. (See Bug #31321 for more information.)
Starting, stopping, or restarting a node may give rise to temporary errors causing some transactions to fail. These include the following cases:
Temporary errors. When first starting a node, it is possible that you may see Error 1204 Temporary failure, distribution changed and similar temporary errors.
Ошибки due to node failure. The stopping or failure of any data node can result in a number of different node failure errors. (However, there should be no aborted transactions when performing a planned shutdown of the cluster.)
In either of these cases, any errors that are generated must be handled within the application. This should be done by retrying the transaction.
See also Section 16.1.6.2, “Limits and Differences of MySQL Cluster from Standard MySQL Limits”.
Some database objects such as tables and indexes have different
limitations when using the
NDBCLUSTER
storage engine:
Number of database objects. The maximum number of all
NDB
database objects in a single MySQL Cluster—including databases, tables, and indexes—is limited to 20320.Attributes per table. The maximum number of attributes (that is, columns and indexes) that can belong to a given table is 512.
Attributes per key. The maximum number of attributes per key is 32.
Row size. The maximum permitted size of any one row is 14000 bytes (as of MySQL Cluster NDB 7.0). Each
BLOB
orTEXT
column contributes 256 + 8 = 264 bytes to this total.Number of rows per partition. A single MySQL Cluster partition can hold a maximum of 46137488 rows. Since the number of partitions is the same as the number of data nodes in the cluster (see Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”), you can increase the available space for data storage by using more data nodes.
In MySQL Cluster NDB 7.0 and later MySQL Cluster release series (including MySQL Cluster NDB 7.2), you can increase the number of data nodes in the cluster while the cluster remains in operation. See Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”, for more information.
It is also possible to increase the number of partitions for
NDB
tables by using explicitKEY
orLINEAR KEY
partitioning (see Section 17.2.5, “KEY
Partitioning”).
A number of features supported by other storage engines are not
supported for NDB
tables. Trying to
use any of these features in MySQL Cluster does not cause errors
in or of itself; however, errors may occur in applications that
expects the features to be supported or enforced:
Foreign key constraints. The foreign key construct is ignored, just as it is in
MyISAM
tables.Index prefixes. Prefixes on indexes are not supported for
NDBCLUSTER
tables. If a prefix is used as part of an index specification in a statement such asCREATE TABLE
,ALTER TABLE
, orCREATE INDEX
, the prefix is ignored.LOAD TABLE ... FROM MASTER
.LOAD TABLE FROM MASTER
is not supported.Savepoints and rollbacks. Savepoints and rollbacks to savepoints are ignored as in
MyISAM
.Durability of commits. There are no durable commits on disk. Commits are replicated, but there is no guarantee that logs are flushed to disk on commit.
Replication. Statement-based replication is not supported. Use
--binlog-format=ROW
(or--binlog-format=MIXED
) when setting up cluster replication. See Section 16.6, “MySQL Cluster Replication”, for more information.
See Section 16.1.6.3, “Limits Relating to Transaction Handling in MySQL Cluster”,
for more information relating to limitations on transaction
handling in NDB
.
The following performance issues are specific to or especially pronounced in MySQL Cluster:
Range scans. There are query performance issues due to sequential access to the
NDB
storage engine; it is also relatively more expensive to do many range scans than it is with eitherMyISAM
orInnoDB
.Reliability of
Records in range
. TheRecords in range
statistic is available but is not completely tested or officially supported. This may result in nonoptimal query plans in some cases. If necessary, you can employUSE INDEX
orFORCE INDEX
to alter the execution plan. See Section 12.2.9.3, “Index Hint Синтаксис”, for more information on how to do this.Unique hash indexes. Unique hash indexes created with
USING HASH
cannot be used for accessing a table ifNULL
is given as part of the key.
The following are limitations specific to the
NDBCLUSTER
storage engine:
Machine architecture. All machines used in the cluster must have the same architecture. That is, all machines hosting nodes must be either big-endian or little-endian, and you cannot use a mixture of both. For example, you cannot have a management node running on a PowerPC which directs a data node that is running on an x86 machine. This restriction does not apply to machines simply running mysql or other clients that may be accessing the cluster's SQL nodes.
Binary logging. MySQL Cluster has the following limitations or restrictions with regard to binary logging:
sql_log_bin
has no effect on data operations; however, it is supported for schema operations.MySQL Cluster cannot produce a binary log for tables having
BLOB
columns but no primary key.Only the following schema operations are logged in a cluster binary log which is not on the mysqld executing the statement:
See also Section 16.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
Disk Data object maxmimums and minimums. Disk data objects are subject to the following maximums and minimums:
Maximum number of tablespaces: 232 (4294967296)
Maximum number of data files per tablespace: 216 (65536)
The theoretical maximum number of extents per tablespace data file is 216 (65536); however, for practical purposes, the recommended maximum number of extents per data file is 215 (32768).
Maximum data file size: The theoretical limit is 64G; however, the practical upper limit is 32G. This is equivalent to 32768 extents of 1M each.
The minimum and maximum possible sizes of extents for tablespace data files are 32K and 2G, respectively. See Section 12.1.18, “
CREATE TABLESPACE
Синтаксис”, for more information.
Disk Data tables and diskless mode. Use of Disk Data tables is not supported when running the cluster in diskless mode. Beginning with MySQL 5.1.12, it is prohibited altogether. (Bug #20008)
Multiple SQL nodes.
The following are issues relating to the use of multiple MySQL
servers as MySQL Cluster SQL nodes, and are specific to the
NDBCLUSTER
storage engine:
No distributed table locks. A
LOCK TABLES
works only for the SQL node on which the lock is issued; no other SQL node in the cluster “sees” this lock. This is also true for a lock issued by any statement that locks tables as part of its operations. (See next item for an example.)ALTER TABLE
operations.ALTER TABLE
is not fully locking when running multiple MySQL servers (SQL nodes). (As discussed in the previous item, MySQL Cluster does not support distributed table locks.)
Multiple management nodes. When using multiple management servers:
You must give nodes explicit IDs in connect strings because automatic allocation of node IDs does not work across multiple management servers.
When a management server starts, it first checks for any other management server in the same MySQL Cluster, and upon successful connection to the other management server uses its configuration data. This means that the management server
--reload
and--initial
startup options are ignored unless the management server is the only one running. It also means that, when performing a rolling restart of a MySQL Cluster with multiple management nodes, the management server reads its own configuration file if (and only if) it is the only management server running in this MySQL Cluster. See Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”, for more information.
Multiple network addresses. Multiple network addresses per data node are not supported. Use of these is liable to cause problems: In the event of a data node failure, an SQL node waits for confirmation that the data node went down but never receives it because another route to that data node remains open. This can effectively make the cluster inoperable.
It is possible to use multiple network hardware
interfaces (such as Ethernet cards) for a
single data node, but these must be bound to the same address.
This also means that it not possible to use more than one
[tcp]
section per connection in the
config.ini
file. See
Section 16.3.2.8, “MySQL Cluster TCP/IP Connections”, for more
information.
A number of limitations and related issues existing in earlier versions of MySQL Cluster have been resolved:
Variable-length column support. The
NDBCLUSTER
storage engine now supports variable-length column types for in-memory tables.Previously, for example, any Cluster table having one or more
VARCHAR
fields which contained only relatively small values, much more memory and disk space were required when using theNDBCLUSTER
storage engine than would have been the case for the same table and data using theMyISAM
engine. In other words, in the case of aVARCHAR
column, such a column required the same amount of storage as aCHAR
column of the same size. In MySQL 5.1, this is no longer the case for in-memory tables, where storage requirements for variable-length column types such asVARCHAR
andBINARY
are comparable to those for these column types when used inMyISAM
tables (see Section 10.5, “Data Type Storage Requirements”).ImportantFor MySQL Cluster Disk Data tables, the fixed-width limitation continues to apply. See Section 16.5.11, “MySQL Cluster Disk Data Tables”.
Replication with MySQL Cluster. It is now possible to use MySQL replication with Cluster databases. For details, see Section 16.6, “MySQL Cluster Replication”.
Circular Replication. Circular replication is also supported with MySQL Cluster, beginning with MySQL 5.1.18. See Section 16.6.10, “MySQL Cluster Replication: Multi-Master and Circular Replication”.
auto_increment_increment
andauto_increment_offset
. Theauto_increment_increment
andauto_increment_offset
server system variables are supported for MySQL Cluster Replication.Backup and restore between architectures. It is possible to perform a Cluster backup and restore between different architectures. Previously—for example—you could not back up a cluster running on a big-endian platform and then restore from that backup to a cluster running on a little-endian system. (Bug #19255)
Multiple data nodes, multi-threaded data nodes. MySQL Cluster NDB 7.2 supports multiple data node processes on a single host as well as multi-threaded data node processes. See Section 16.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
Identifiers. Formerly (in MySQL 5.0 and earlier), database names, table names and attribute names could not be as long for
NDB
tables as tables using other storage engines, because attribute names were truncated internally. In MySQL 5.1 and later, names of MySQL Cluster databases, tables, and table columns follow the same rules regarding length as they do for any other storage engine.Length of
CREATE TABLE
statements.CREATE TABLE
statements may be no more than 4096 characters in length. This limitation affects MySQL 5.1.6, 5.1.7, and 5.1.8 only. (See Bug #17813)IGNORE
andREPLACE
functionality. In MySQL 5.1.7 and earlier,INSERT IGNORE
,UPDATE IGNORE
, andREPLACE
were supported only for primary keys, but not for unique keys. It was possible to work around this issue by removing the constraint, then dropping the unique index, performing any inserts, and then adding the unique index again.This limitation was removed for
INSERT IGNORE
andREPLACE
in MySQL 5.1.8. (See Bug #17431.)AUTO_INCREMENT
columns. In MySQL 5.1.10 and earlier versions, the maximum number of tables havingAUTO_INCREMENT
columns—including those belonging to hidden primary keys—was 2048.This limitation was lifted in MySQL 5.1.11.
Maximum number of cluster nodes. The total maximum number of nodes in a MySQL Cluster is 255, including all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers. The total number of data nodes and management nodes is 63, of which up to 48 can be data nodes.
ЗамечаниеA data node cannot have a node ID greater than 49.
Recovery of memory from deleted rows. Memory can be reclaimed from an
NDB
table for reuse with anyNDB
table by employingOPTIMIZE TABLE
, subject to the following limitations:Only in-memory tables are supported; the
OPTIMIZE TABLE
statement has no effect on MySQL Cluster Disk Data tables.Only variable-length columns (such as those declared as
VARCHAR
,TEXT
, orBLOB
) are supported.However, you can force columns defined using fixed-length data types (such as
CHAR
) to be dynamic using theROW_FORMAT
orCOLUMN_FORMAT
option with aCREATE TABLE
orALTER TABLE
statement.See Section 12.1.17, “
CREATE TABLE
Синтаксис”, and Section 12.1.7, “ALTER TABLE
Синтаксис”, for information on these options.
You can regulate the effects of
OPTIMIZE
on performance by adjusting the value of the global system variablendb_optimization_delay
, which sets the number of milliseconds to wait between batches of rows being processed byOPTIMIZE
. The default value is 10 milliseconds. It is possible to set a lower value (to a minimum of0
), but not recommended. The maximum is 100000 milliseconds (that is, 100 seconds).Number of tables. The maximum number of
NDBCLUSTER
tables in a single MySQL Cluster is included in the total maximum number ofNDBCLUSTER
database objects (20320). (See Section 16.1.6.5, “Limits Associated with Database Objects in MySQL Cluster”.)Adding and dropping of data nodes. In MySQL Cluster NDB 7.2 (MySQL Cluster NDB 7.0 and later), it is possible to add new data nodes to a running MySQL Cluster by performing a rolling restart, so that the cluster and the data stored in it remain available to applications.
When planning to increase the number of data nodes in the cluster online, you should be aware of and take into account the following issues:
New data nodes can be added online to a MySQL Cluster only as part of a new node group.
New data nodes can be added online, but cannot be dropped online. Reducing the number of data nodes requires a system restart of the cluster.
As in previous MySQL Cluster releases, it is not possible to change online either the number of replicas (
NoOfReplicas
configuration parameter) or the number of data nodes per node group. These changes require a system restart.Redistribution of existing cluster data using the new data nodes is not automatic; however, this can be accomplished using simple SQL statements in the mysql client or other MySQL client application once the nodes have been added. During this procedure, it is not possible to perform DDL operations, although DML operations can continue as normal.
The distribution of new cluster data (that is, data stored in the cluster after the new nodes have been added) uses the new nodes without manual intervention.
For more information, see Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”.
Distribution of MySQL users and privileges. Previously, MySQL users and privileges created on one SQL node were unique to that SQL node, due to the fact that the MySQL grant tables were restricted to using the
MyISAM
storage engine. Beginning with MySQL Cluster NDB 7.2.0, it is possible, following installation of the MySQL Cluster software and setup of the desired users and privileges on one SQL node, to convert the grant tables to useNDB
and thus to distribute the users and privileges across all SQL nodes connected to the cluster. You can do this by loading and making use of a set of stored procedures defined in an SQL script supplied with the MySQL Cluster distribution. For more information, see Section 16.5.13, “Distributed MySQL Privileges for MySQL Cluster”.
- 16.2.1. Installing MySQL Cluster on Linux
- 16.2.2. Installing MySQL Cluster on Windows
- 16.2.3. Initial Configuration of MySQL Cluster
- 16.2.4. Initial Startup of MySQL Cluster
- 16.2.5. MySQL Cluster Пример with Tables and Data
- 16.2.6. Safe Shutdown and Restart of MySQL Cluster
- 16.2.7. Upgrading and Downgrading MySQL Cluster NDB 7.2
This section describes the basics for planning, installing, configuring, and running a MySQL Cluster. Whereas the examples in Section 16.3, “MySQL Cluster Configuration” provide more in-depth information on a variety of clustering options and configuration, the result of following the guidelines and procedures outlined here should be a usable MySQL Cluster which meets the minimum requirements for availability and safeguarding of data.
For information about upgrading or downgrading a MySQL Cluster between release versions, see Section 16.2.7, “Upgrading and Downgrading MySQL Cluster NDB 7.2”.
This section covers hardware and software requirements; networking issues; installation of MySQL Cluster; configuration issues; starting, stopping, and restarting the cluster; loading of a sample database; and performing queries.
Assumptions. The following sections make a number of assumptions regarding the cluster's physical and network configuration. These assumptions are discussed in the next few paragraphs.
Cluster nodes and host computers. The cluster consists of four nodes, each on a separate host computer, and each with a fixed network address on a typical Ethernet network as shown here:
Node | IP Address |
---|---|
Management node (mgmd) | 192.168.0.10 |
SQL node (mysqld) | 192.168.0.20 |
Data node "A" (ndbd) | 192.168.0.30 |
Data node "B" (ndbd) | 192.168.0.40 |
This may be made clearer by the following diagram:

Network addressing.
In the interest of simplicity (and reliability), this
How-To uses only numeric IP addresses.
However, if DNS resolution is available on your network, it is
possible to use host names in lieu of IP addresses in configuring
Cluster. Alternatively, you can use the hosts
file (typically /etc/hosts
for Linux and
other Unix-like operating systems,
C:\WINDOWS\system32\drivers\etc\hosts
on
Windows, or your operating system's equivalent) for providing
a means to do host lookup if such is available.
Potential hosts file issues.
A common problem when trying to use host names for Cluster nodes
arises because of the way in which some operating systems
(including some Linux distributions) set up the system's own
host name in the /etc/hosts
during
installation. Consider two machines with the host names
ndb1
and ndb2
, both in the
cluster
network domain. Red Hat Linux
(including some derivatives such as CentOS and Fedora) places the
following entries in these machines'
/etc/hosts
files:
# ndb1 /etc/hosts
:
127.0.0.1 ndb1.cluster ndb1 localhost.localdomain localhost
# ndb2 /etc/hosts
:
127.0.0.1 ndb2.cluster ndb2 localhost.localdomain localhost
SUSE Linux (including OpenSUSE) places these entries in the
machines' /etc/hosts
files:
# ndb1 /etc/hosts
:
127.0.0.1 localhost
127.0.0.2 ndb1.cluster ndb1
# ndb2 /etc/hosts
:
127.0.0.1 localhost
127.0.0.2 ndb2.cluster ndb2
In both instances, ndb1
routes
ndb1.cluster
to a loopback IP address, but gets a
public IP address from DNS for ndb2.cluster
,
while ndb2
routes ndb2.cluster
to a loopback address and obtains a public address for
ndb1.cluster
. The result is that each data node
connects to the management server, but cannot tell when any other
data nodes have connected, and so the data nodes appear to hang
while starting.
You cannot mix localhost
and other host names
or IP addresses in config.ini
. For these
reasons, the solution in such cases (other than to use IP
addresses for all
config.ini
HostName
entries) is to remove the fully qualified host names from
/etc/hosts
and use these in
config.ini
for all cluster hosts.
Host computer type. Each host computer in our installation scenario is an Intel-based desktop PC running a supported operating system installed to disk in a standard configuration, and running no unnecessary services. The core operating system with standard TCP/IP networking capabilities should be sufficient. Also for the sake of simplicity, we also assume that the file systems on all hosts are set up identically. In the event that they are not, you should adapt these instructions accordingly.
Network hardware. Standard 100 Mbps or 1 gigabit Ethernet cards are installed on each machine, along with the proper drivers for the cards, and that all four hosts are connected through a standard-issue Ethernet networking appliance such as a switch. (All machines should use network cards with the same throughout. That is, all four machines in the cluster should have 100 Mbps cards or all four machines should have 1 Gbps cards.) MySQL Cluster works in a 100 Mbps network; however, gigabit Ethernet provides better performance.
MySQL Cluster is not intended for use in a network for which throughput is less than 100 Mbps or which experiences a high degree of latency. For this reason (among others), attempting to run a MySQL Cluster over a wide area network such as the Internet is not likely to be successful, and is not supported in production.
Sample data.
We use the world
database which is available
for download from the MySQL Web site (see
http://dev.mysql.com/doc/index-other.html). We assume that
each machine has sufficient memory for running the operating
system, required MySQL Cluster processes, and (on the data nodes)
storing the database.
For general information about installing MySQL, see Глава 2, Installing and Upgrading MySQL. For information about installation of MySQL Cluster on Linux and other Unix-like operating systems, see Section 16.2.1, “Installing MySQL Cluster on Linux”. For information about installation of MySQL Cluster on Windows operating systems, see Section 16.2.2, “Installing MySQL Cluster on Windows”.
For general information about MySQL Cluster hardware, software, and networking requirements, see Section 16.1.3, “MySQL Cluster Hardware, Software, and Networking Requirements”.
This section covers installation of MySQL Cluster on Linux and other Unix-like operating systems. While the next few sections refer to a Linux operating system, the instructions and procedures given there should be easily adaptable to other supported Unix-like platforms.
MySQL Cluster NDB 7.2 is also available for Windows operating systems; for installation and setup instructions specific to Windows, see Section 16.2.2, “Installing MySQL Cluster on Windows”.
Each MySQL Cluster host computer must have the correct executable programs installed. A host running an SQL node must have installed on it a MySQL Server binary (mysqld). Management nodes require the management server daemon (ndb_mgmd); data nodes require the data node daemon (ndbd or ndbmtd). It is not necessary to install the MySQL Server binary on management node hosts and data node hosts. It is recommended that you also install the management client (ndb_mgm) on the management server host.
Installation of MySQL Cluster on Linux can be done using precompiled binaries from Oracle (downloaded as a .tar.gz archive), with RPM packages (also available from Oracle), or from source code. All three of these installation methods are described in the section that follow.
Regardless of the method used, it is still necessary following installation of the MySQL Cluster binaries to create configuration files for all cluster nodes, before you can start the cluster. See Section 16.2.3, “Initial Configuration of MySQL Cluster”.
This section covers the steps necessary to install the correct executables for each type of Cluster node from precompiled binaries supplied by Oracle.
For setting up a cluster using precompiled binaries, the first
step in the installation process for each cluster host is to
download the latest MySQL Cluster NDB 7.2 binary archive
(mysql-cluster-gpl-7.2.5-linux-i686-glibc23.tar.gz
)
from the MySQL Cluster
downloads area. We assume that you have placed this file
in each machine's /var/tmp
directory.
(If you do require a custom binary, see
Section 2.9.3, “Installing MySQL from a Development Source Tree”.)
After completing the installation, do not yet start any of the binaries. We show you how to do so following the configuration of the nodes (see Section 16.2.3, “Initial Configuration of MySQL Cluster”).
Data nodes and SQL nodes.
On each of the machines designated to host data nodes or SQL
nodes, perform the following steps as the system
root
user:
Check your
/etc/passwd
and/etc/group
files (or use whatever tools are provided by your operating system for managing users and groups) to see whether there is already amysql
group andmysql
user on the system. Some OS distributions create these as part of the operating system installation process. If they are not already present, create a newmysql
user group, and then add amysql
user to this group:shell>
groupadd mysql
shell>useradd -g mysql mysql
The syntax for useradd and groupadd may differ slightly on different versions of Unix, or they may have different names such as adduser and addgroup.
Change location to the directory containing the downloaded file, unpack the archive, and create a symbolic link named
mysql
to themysql
directory. Note that the actual file and directory names vary according to the MySQL Cluster version number.shell>
cd /var/tmp
shell>tar -C /usr/local -xzvf mysql-cluster-gpl-7.1.20-linux-i686-glibc23.tar.gz
shell>ln -s /usr/local/mysql-cluster-gpl-7.1.20-linux-i686-glibc23 /usr/local/mysql
Change location to the
mysql
directory and run the supplied script for creating the system databases:shell>
cd mysql
shell>scripts/mysql_install_db --user=mysql
Set the necessary permissions for the MySQL server and data directories:
shell>
chown -R root .
shell>chown -R mysql data
shell>chgrp -R mysql .
Note that the data directory on each machine hosting a data node is
/usr/local/mysql/data
. This piece of information is essential when configuring the management node. (See Section 16.2.3, “Initial Configuration of MySQL Cluster”.)Copy the MySQL startup script to the appropriate directory, make it executable, and set it to start when the operating system is booted up:
shell>
cp support-files/mysql.server /etc/rc.d/init.d/
shell>chmod +x /etc/rc.d/init.d/mysql.server
shell>chkconfig --add mysql.server
(The startup scripts directory may vary depending on your operating system and version—for example, in some Linux distributions, it is
/etc/init.d
.)Here we use Red Hat's chkconfig for creating links to the startup scripts; use whatever means is appropriate for this purpose on your operating system and distribution, such as update-rc.d on Debian.
Remember that the preceding steps must be repeated on each machine where an SQL node is to reside.
Management nodes.
Installation of the management node does not require the
mysqld binary. Only the MySQL Cluster
management server (ndb_mgmd) is required;
you most likely want to install the management client
(ndb_mgm) as well. Both of these binaries
also be found in the .tar.gz
archive.
Again, we assume that you have placed this archive in
/var/tmp
.
As system root
(that is, after using
sudo, su root, or your
system's equivalent for temporarily assuming the system
administrator account's privileges), perform the following steps
to install ndb_mgmd and
ndb_mgm on the Cluster management node host:
Change location to the
/var/tmp
directory, and extract the ndb_mgm and ndb_mgmd from the archive into a suitable directory such as/usr/local/bin
:shell>
cd /var/tmp
shell>tar -zxvf mysql-5.1.61-ndb-7.1.20-linux-i686-glibc23.tar.gz
shell>cd mysql-5.1.61-ndb-7.1.20-linux-i686-glibc23
shell>cp bin/ndb_mgm* /usr/local/bin
(You can safely delete the directory created by unpacking the downloaded archive, and the files it contains, from
/var/tmp
once ndb_mgm and ndb_mgmd have been copied to the executables directory.)Change location to the directory into which you copied the files, and then make both of them executable:
shell>
cd /usr/local/bin
shell>chmod +x ndb_mgm*
In Section 16.2.3, “Initial Configuration of MySQL Cluster”, we create configuration files for all of the nodes in our example MySQL Cluster.
This section covers the steps necessary to install the correct executables for each type of MySQL Cluster node using RPM packages supplied by Oracle.
RPMs are available for both 32-bit and 64-bit Linux platforms. For a MySQL Cluster, three RPMs are required:
The Server RPM (for example,
MySQL-Cluster-gpl-server-6.3.48-0.sles10.i586.rpm
,MySQL-Cluster-gpl-server-7.0.31-0.sles10.i586.rpm
, orMySQL-Cluster-gpl-server-7.1.20-0.sles10.i586.rpm
), which supplies the core files needed to run a MySQL Server withNDBCLUSTER
storage engine support (that is, as a MySQL Cluster SQL node).If you do not have your own client application capable of administering a MySQL server, you should also obtain and install the Client RPM (for example,
MySQL-Cluster-gpl-client-6.3.48-0.sles10.i586.rpm
,MySQL-Cluster-gpl-client-7.0.31-0.sles10.i586.rpm
, orMySQL-Cluster-gpl-client-7.1.20-0.sles10.i586.rpm
).The Cluster storage engine RPM (for example,
MySQL-Cluster-gpl-storage-6.3.48-0.sles10.i586.rpm
,MySQL-Cluster-gpl-storage-7.0.31-0.sles10.i586.rpm
, orMySQL-Cluster-gpl-storage-7.1.20-0.sles10.i586.rpm
), which supplies the MySQL Cluster data node binary (ndbd).The Cluster storage engine management RPM (for example,
MySQL-Cluster-gpl-management-6.3.48-0.sles10.i586.rpm
,MySQL-Cluster-gpl-management-7.0.31-0.sles10.i586.rpm
, orMySQL-Cluster-gpl-management-7.1.20-0.sles10.i586.rpm
) which provides the MySQL Cluster management server binary (ndb_mgmd).
In addition, you should also obtain the
NDB Cluster - Storage engine basic
tools RPM (for example,
MySQL-Cluster-gpl-tools-6.3.48-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-tools-7.0.31-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-tools-7.1.20-0.sles10.i586.rpm
),
which supplies several useful applications for working with a
MySQL Cluster. The most important of these is the MySQL Cluster
management client (ndb_mgm). The
NDB Cluster - Storage engine extra
tools RPM (for example,
MySQL-Cluster-gpl-extra-6.3.48-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-extra-7.0.31-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-extra-7.1.20-0.sles10.i586.rpm
)
contains some additional testing and monitoring programs, but is
not required to install a MySQL Cluster. (For more information
about these additional programs, see
Section 16.4, “MySQL Cluster Programs”.)
The MySQL Cluster version number in the RPM file names (shown
here as 6.3.48
,
7.0.31
, or
7.1.20
) can vary according
to the version which you are actually using. It is
very important that all of the Cluster RPMs to be installed have
the same version number. The glibc
version number (if present), and architecture designation (shown
here as i586
) should be appropriate to the
machine on which the RPM is to be installed.
Data nodes. On a computer that is to host a cluster data node it is necessary to install only the NDB Cluster - Storage engine RPM. To do so, copy this RPM to the data node host, and run the following command as the system root user, replacing the name shown for the RPM as necessary to match that of the RPM downloaded from the MySQL web site:
shell> rpm -Uhv MySQL-Cluster-gpl-storage-7.1.20-0.sles10.i586.rpm
The previous command installs the MySQL Cluster data node binary
(ndbd) in the /usr/sbin
directory.
SQL nodes. On each machine to be used for hosting a cluster SQL node, install the Server RPM by executing the following command as the system root user, replacing the name shown for the RPM as necessary to match the name of the RPM downloaded from the MySQL web site:
shell> rpm -Uhv MySQL-Cluster-gpl-server-7.1.20-0.sles10.i586.rpm
This installs the MySQL server binary
(mysqld) in the
/usr/sbin
directory, as well as all needed
MySQL Server support files. It also installs the
mysql.server and
mysqld_safe startup scripts in
/usr/share/mysql
and
/usr/bin
, respectively. The RPM installer
should take care of general configuration issues (such as
creating the mysql
user and group, if needed)
automatically.
To administer the SQL node (MySQL server), you should also install the Client RPM, as shown here:
shell> rpm -Uhv MySQL-Cluster-gpl-client-7.1.20-0.sles10.i586.rpm
This installs the mysql client program.
Management nodes. To install the MySQL Cluster management server, it is necessary only to use the NDB Cluster - Storage engine management RPM. Copy this RPM to the computer intended to host the management node, and then install it by running the following command as the system root user (replace the name shown for the RPM as necessary to match that of the Storage engine management RPM downloaded from the MySQL web site):
shell> rpm -Uhv MySQL-Cluster-gpl-management-7.1.20-0.sles10.i586.rpm
This installs the management server binary
(ndb_mgmd) to the
/usr/sbin
directory.
You should also install the NDB
management client, which is supplied by the
Storage engine basic tools RPM.
Copy this RPM to the same computer as the management node, and
then install it by running the following command as the system
root user (again, replace the name shown for the RPM as
necessary to match that of the Storage
engine basic tools RPM downloaded from the MySQL web
site):
shell> rpm -Uhv MySQL-Cluster-gpl-tools-7.1.20-0.sles10.i586.rpm
The Storage engine basic tools
RPM installs the MySQL Cluster management client
(ndb_mgm) to the
/usr/bin
directory.
You can also install the Cluster storage engine extra tools RPM, if you wish, as shown here:
shell> rpm -Uhv MySQL-Cluster-gpl-extra-7.1.20-0.sles10.i586.rpm
You may find the extra tools useful; however the Cluster storage engine extra tools RPM is not required to install a working MySQL Cluster.
See Section 2.5.1, “Installing MySQL from RPM Packages on Linux”, for general information about installing MySQL using RPMs supplied by Oracle.
After installing from RPM, you still need to configure the cluster as discussed in Section 16.2.3, “Initial Configuration of MySQL Cluster”.
This section provides information about compiling MySQL Clusteron Linux and other Unix-like platforms. Building MySQL Cluster from source is similar to building the standard MySQL Server, although it differs in a few key respects discussed here. For general information about building MySQL from source, see Section 2.9, “Installing MySQL from Source”. For information about compiling MySQL Cluster on Windows platforms, see Section 16.2.2.2, “Compiling and Installing MySQL Cluster from Source on Windows”.
Building MySQL Cluster requires using the MySQL Cluster sources.
These are available from the MySQL Cluster downloads page at
http://dev.mysql.com/downloads/cluster/
. The archived source
file should have a name similar to
mysql-cluster-gpl-7.2.5.tar.gz
.
You can also obtain MySQL development sources from
launchpad.net
. Attempting to build MySQL Cluster from
standard MySQL Server 5.5 sources is not supported.
The WITH_NDBCLUSTER_STORAGE_ENGINE
option for CMake causes the binaries for the
management nodes, data nodes, and other MySQL Cluster programs
to be built; it also causes mysqld to be
compiled with NDB
storage engine
support. This option is enabled by default in the MySQL Cluster
NDB 7.2 sources.
For more information about CMake options specific to building MySQL Cluster, see Options for Compiling MySQL Cluster.
After you have run make && make install (or your system's equivalent), the result is similar to what is obtained by unpacking a precompiled binary to the same location.
Management nodes.
When building from source and running the default
make install, the management server and
management client binaries (ndb_mgmd and
ndb_mgm) can be found in
/usr/local/mysql/bin
. Only
ndb_mgmd is required to be present on a
management node host; however, it is also a good idea to have
ndb_mgm present on the same host machine.
Neither of these executables requires a specific location on
the host machine's file system.
Data nodes.
The only executable required on a data node host is the data
node binary ndbd or
ndbmtd. (mysqld, for
example, does not have to be present on the host machine.) By
default, when building from source, this file is placed in the
directory /usr/local/mysql/bin
. For
installing on multiple data node hosts, only
ndbd or ndbmtd need be
copied to the other host machine or machines. (This assumes
that all data node hosts use the same architecture and
operating system; otherwise you may need to compile separately
for each different platform.) The data node binary need not be
in any particular location on the host's file system, as long
as the location is known.
When compiling MySQL Cluster from source, no special options are
required for building multi-threaded data node binaries.
Configuring the build with NDB
storage engine support causes ndbmtd to be
built automatically; make install places the
ndbmtd binary in the installation
bin
directory along with
mysqld, ndbd, and
ndb_mgm.
SQL nodes.
If you compile MySQL with clustering support, and perform the
default installation (using make install as
the system root
user),
mysqld is placed in
/usr/local/mysql/bin
. Follow the steps
given in Section 2.9, “Installing MySQL from Source” to make
mysqld ready for use. If you want to run
multiple SQL nodes, you can use a copy of the same
mysqld executable and its associated
support files on several machines. The easiest way to do this
is to copy the entire /usr/local/mysql
directory and all directories and files contained within it to
the other SQL node host or hosts, then repeat the steps from
Section 2.9, “Installing MySQL from Source” on each machine. If you
configure the build with a nondefault PREFIX
option, you must adjust the directory accordingly.
In Section 16.2.3, “Initial Configuration of MySQL Cluster”, we create configuration files for all of the nodes in our example MySQL Cluster.
MySQL Cluster NDB 7.2 binaries for Windows can be obtained from http://dev.mysql.com/downloads/cluster/. For information about installing MySQL Cluster on Windows from a binary release provided by Oracle, see Section 16.2.2.1, “Installing MySQL Cluster on Windows from a Binary Release”.
It is also possible to compile and install MySQL Cluster from source on Windows using Microsoft Visual Studio. For more information, see Section 16.2.2.2, “Compiling and Installing MySQL Cluster from Source on Windows”.
This section describes a basic installation of MySQL Cluster on
Windows using a binary no-install
MySQL
Cluster release provided by Oracle, using the same 4-node setup
outlined in the beginning of this section (see
Section 16.2, “MySQL Cluster Installation”), as shown in the
following table:
Node | IP Address |
---|---|
Management (MGMD) node | 192.168.0.10 |
MySQL server (SQL) node | 192.168.0.20 |
Data (NDBD) node "A" | 192.168.0.30 |
Data (NDBD) node "B" | 192.168.0.40 |
As on other platforms, the MySQL Cluster host computer running an SQL node must have installed on it a MySQL Server binary (mysqld.exe). You should also have the MySQL client (mysql.exe) on on this host. For management nodes and data nodes, it is not necessary to install the MySQL Server binary; however, each management node requires the management server daemon (ndb_mgmd.exe); each data node requires the data node daemon (ndbd.exe or ndbmtd.exe). For this example, we refer to ndbd.exe as the data node executable, but you can install ndbmtd.exe, the multi-threaded version of this program, instead, in exactly the same way. You should also install the management client (ndb_mgm.exe) on the management server host. This section covers the steps necessary to install the correct Windows binaries for each type of MySQL Cluster node.
As with other Windows programs, MySQL Cluster executables are
named with the .exe
file extension.
However, it is not necessary to include the
.exe
extension when invoking these
programs from the command line. Therefore, we often simply
refer to these programs in this documentation as
mysqld, mysql,
ndb_mgmd, and so on. You should understand
that, whether we refer (for example) to
mysqld or mysqld.exe,
either name means the same thing (the MySQL Server program).
For setting up a MySQL Cluster using Oracles's
no-install
binaries, the first step in the
installation process is to download the latest MySQL Cluster
Windows binary archive from
http://dev.mysql.com/downloads/cluster/. This archive has a
filename of the form
mysql-cluster-gpl-noinstall-
,
where ver
-winarch
.zipver
is the
NDB
storage engine version (such as
7.2.1
), and arch
is the architecture (32
for 32-bit binaries,
and 64
for 64-bit binaries). For example, the
MySQL Cluster NDB 7.2.1 no-install
archive
for 32-bit Windows systems is named
mysql-cluster-gpl-noinstall-7.2.1-win32.zip
.
You can run 32-bit MySQL Cluster binaries on both 32-bit and 64-bit versions of Windows; however, 64-bit MySQL Cluster binaries can be used only on 64-bit versions of Windows. If you are using a 32-bit version of Windows on a computer that has a 64-bit CPU, then you must use the 32-bit MySQL Cluster binaries.
To minimize the number of files that need to be downloaded from the Internet or copied between machines, we start with the computer where you intend to run the SQL node.
SQL node.
We assume that you have placed a copy of the
no-install
archive in the directory
C:\Documents and
Settings\
on the computer having the IP
address 192.168.0.20, where
username
\My
Documents\Downloadsusername
is the name of the current
user. (You can obtain this name using ECHO
%USERNAME%
on the command line.) To install and run
MySQL Cluster executables as Windows services, this user
should be a member of the Administrators
group.
Extract all the files from the archive. The Extraction Wizard
integrated with Windows Explorer is adequate for this task. (If
you use a different archive program, be sure that it extracts
all files and directories from the archive, and that it
preserves the archive's directory structure.) When you are
asked for a destination directory, enter
C:\
, which causes the Extraction Wizard to
extract the archive to the directory
C:\mysql-cluster-gpl-noinstall-
.
Rename this directory to ver
-winarch
C:\mysql
.
It is possible to install the MySQL Cluster binaries to
directories other than C:\mysql\bin
;
however, if you do so, you must modify the paths shown in this
procedure accordingly. In particular, if the MySQL Server (SQL
node) binary is installed to a location other than
C:\mysql
or C:\Program
Files\MySQL\MySQL Server 5.5
, or if the
SQL node's data directory is in a location other than
C:\mysql\data
or C:\Program
Files\MySQL\MySQL Server 5.5\data
, extra
configuration options must be used on the command line or added
to the my.ini
or
my.cnf
file when starting the SQL node. For
more information about configuring a MySQL Server to run in a
nonstandard location, see
Section 2.3.6, “Installing MySQL on Microsoft Windows Using a
noinstall
Zip Archive”.
For a MySQL Server with MySQL Cluster support to run as part of
a MySQL Cluster, it must be started with the options
--ndbcluster
and
--ndb-connectstring
. While you
can specify these options on the command line, it is usually
more convenient to place them in an option file. To do this,
create a new text file in Notepad or another text editor. Enter
the following configuration information into this file:
[mysqld] # Options for mysqld process: ndbcluster # run NDB storage engine ndb-connectstring=192.168.0.10 # location of management server
You can add other options used by this MySQL Server if desired
(see Section 2.3.6.2, “Creating an Option File”), but the file
must contain the options shown, at a minimum. Save this file as
C:\mysql\my.ini
. This completes the
installation and setup for the SQL node.
Data nodes.
A MySQL Cluster data node on a Windows host requires only a
single executable, one of either ndbd.exe
or ndbmtd.exe. For this example, we assume
that you are using ndbd.exe, but the same
instructions apply when using ndbmtd.exe.
On each computer where you wish to run a data node (the
computers having the IP addresses 192.168.0.30 and
192.168.0.40), create the directories
C:\mysql
,
C:\mysql\bin
, and
C:\mysql\cluster-data
; then, on the
computer where you downloaded and extracted the
no-install
archive, locate
ndbd.exe
in the
C:\mysql\bin
directory. Copy this file to
the C:\mysql\bin
directory on each of the
two data node hosts.
To function as part of a MySQL Cluster, each data node must be
given the address or hostname of the management server. You can
supply this information on the command line using the
--ndb-connectstring
or
-c
option when starting each data node process.
However, it is usually preferable to put this information in an
option file. To do this, create a new text file in Notepad or
another text editor and enter the following text:
[mysql_cluster] # Options for data node process: ndb-connectstring=192.168.0.10 # location of management server
Save this file as C:\mysql\my.ini
on the
data node host. Create another text file containing the same
information and save it on as
C:mysql\my.ini
on the other data node host,
or copy the my.ini file from the first data node host to the
second one, making sure to place the copy in the second data
node's C:\mysql
directory. Both data
node hosts are now ready to be used in the MySQL Cluster, which
leaves only the management node to be installed and configured.
Management node.
The only executable program required on a computer used for
hosting a MySQL Cluster management node is the management
server program ndb_mgmd.exe. However, in
order to administer the MySQL Cluster once it has been
started, you should also install the MySQL Cluster management
client program ndb_mgm.exe on the same
machine as the management server. Locate these two programs on
the machine where you downloaded and extracted the
no-install
archive; this should be the
directory C:\mysql\bin
on the SQL node
host. Create the directory C:\mysql\bin
on the computer having the IP address 192.168.0.10, then copy
both programs to this directory.
You should now create two configuration files for use by
ndb_mgmd.exe
:
A local configuration file to supply configuration data spcific to the management node itself. Typically, this file needs only to supply the location of the MySQL Cluster global configuration file (see item 2).
To create this file, start a new text file in Notepad or another text editor, and enter the following information:
[mysql_cluster] # Options for management node process config-file=C:/mysql/bin/config.ini
Save this file as the plaintext file
C:\mysql\bin\my.ini
.A global configuration file from which the management node can obtain configuration information governing the MySQL Cluster as a whole. At a minimum, this file must contain a section for each node in the MySQL Cluster, and the IP addresses or hostnames for the management node and all data nodes (
HostName
configuration parameter). It is also advisable to include the following additional information:The IP address or hostname of any SQL nodes
The data memory and index memory allocated to each data node (
DataMemory
andIndexMemory
configuration parameters)The number of replicas, using the
NoOfReplicas
configuration parameter (see Section 16.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”)The directory where each data node stores it data and log file, and the directory where the management node keeps its log files (in both cases, the
DataDir
configuration parameter)
Create a new text file using a text editor such as Notepad, and input the following information:
[ndbd default] # Options affecting ndbd processes on all data nodes: NoOfReplicas=2 # Number of replicas DataDir=C:/mysql/bin/cluster-data # Directory for each data node's data files # Forward slashes used in directory path, # rather than backslashes. This is correct; # see Important note in text DataMemory=80M # Memory allocated to data storage IndexMemory=18M # Memory allocated to index storage # For DataMemory and IndexMemory, we have used the # default values. Since the "world" database takes up # only about 500KB, this should be more than enough for # this example Cluster setup. [ndb_mgmd] # Management process options: HostName=192.168.0.10 # Hostname or IP address of management node DataDir=C:/mysql/bin/cluster-logs # Directory for management node log files [ndbd] # Options for data node "A": # (one [ndbd] section per data node) HostName=192.168.0.30 # Hostname or IP address [ndbd] # Options for data node "B": HostName=192.168.0.40 # Hostname or IP address [mysqld] # SQL node options: HostName=192.168.0.20 # Hostname or IP address
Save this file as the plaintext file
C:\mysql\bin\config.ini
.
A single backslash character (\
) cannot be
used when specifying directory paths in program options or
configuration files used by MySQL Cluster on Windows. Instead,
you must either escape each backslash character with a second
backslash (\\
), or replace the backslash
with a forward slash character (/
). For
example, the following line from the
[ndb_mgmd]
section of a MySQL Cluster
config.ini
file does not work:
DataDir=C:\mysql\bin\cluster-logs
Instead, you may use either of the following:
DataDir=C:\\mysql\\bin\\cluster-logs # Escaped backslashes
DataDir=C:/mysql/bin/cluster-logs # Forward slashes
For reasons of brevity and legibility, we recommend that you use forward slashes in directory paths used in MySQL Cluster program options and configuration files on Windows.
Oracle provides precompiled MySQL Cluster binaries for Windows which should be adequate for most users. However, if you wish, it is also possible to compile MySQL Cluster for Windows from source code. The procedure for doing this is almost identical to the procedure used to compile the standard MySQL Server binaries for Windows, and uses the same tools. However, there are two major differences:
To build MySQL Cluster, you must use the MySQL Cluster sources, which you can obtain from http://dev.mysql.com/downloads/cluster/.
Attempting to build MySQL Cluster from the source code for the standard MySQL Server is likely not to be successful, and is not supported by Oracle.
You must configure the build using the
WITH_NDBCLUSTER_STORAGE_ENGINE
option in addition to any other build options you wish to use before creating the Visual Studio project files. Once you have run configure.js with the desired options, you can create the project files and build from them in the same manner as you do when compiling the standard MySQL Server. For more information, see Installing MySQL from Source on Windows.
Once the build process is complete, you can create a Zip archive
containing the compiled binaries by executing make
package. The MySQL Cluster binaries can be found in
the bin
directory of the resulting archive,
which is equivalent to the no-install
archive, and which can be installed and configured in the same
manner. For more information, see
Section 16.2.2.1, “Installing MySQL Cluster on Windows from a Binary Release”.
Once the MySQL Cluster executables and needed configuration files are in place, performing an initial start of the cluster is simply a matter of starting the MySQL Cluster executables for all nodes in the cluster. Each cluster node process must be started separately, and on the host computer where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes.
On the management node host, issue the following command from the command line to start the management node process:
C:\mysql\bin>
ndb_mgmd
2010-06-23 07:53:34 [MgmtSrvr] INFO -- NDB Cluster Management Server. mysql-5.1.61-ndb-7.1.20 2010-06-23 07:53:34 [MgmtSrvr] INFO -- Reading cluster configuration from 'config.ini'The management node process continues to print logging output to the console. This is normal, because the management node is not running as a Windows service. (If you have used MySQL Cluster on a Unix-like platform such as Linux, you may notice that the management node's default behavior in this regard on Windows is effectively the opposite of its behavior on Unix systems, where it runs by default as a Unix daemon process. This behavior is also true of MySQL Cluster data node processes running on Windows.) For this reason, do not close the window in which ndb_mgmd.exe is running; doing so kills the management node process. (See Section 16.2.2.4, “Installing MySQL Cluster Processes as Windows Services”, where we show how to install and run MySQL Cluster processes as Windows services.)
The required
-f
option tells the management node where to find the global configuration file (config.ini
). The long form of this option is--config-file
.ImportantA MySQL Cluster management node caches the configuration data that it reads from
config.ini
; once it has created a configuration cache, it ignores theconfig.ini
file on subsequent starts unless forced to do otherwise. This means that, if the management node fails to start due to an error in this file, you must make the management node re-readconfig.ini
after you have corrected any errors in it. You can do this by starting ndb_mgmd.exe with the--reload
or--initial
option on the command line. Either of these options works to refresh the configuration cache.It is not necessary or advisable to use either of these options in the management node's
my.ini
file.For additional information about options which can be used with ndb_mgmd, see Section 16.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”, as well as Section 16.4.23, “Options Common to MySQL Cluster Programs”.
On each of the data node hosts, run the command shown here to start the data node processes:
C:\mysql\bin>
ndbd
2010-06-23 07:53:46 [ndbd] INFO -- Configuration fetched from 'localhost:1186', generation: 1In each case, the first line of output from the data node process should resemble what is shown in the preceding example, and is followed by additional lines of logging output. As with the management node process, this is normal, because the data node is not running as a Windows service. For this reason, do not close the console window in which the data node process is running; doing so kills ndbd.exe. (For more information, see Section 16.2.2.4, “Installing MySQL Cluster Processes as Windows Services”.)
Do not start the SQL node yet; it cannot connect to the cluster until the data nodes have finished starting, which may take some time. Instead, in a new console window on the management node host, start the MySQL Cluster management client ndb_mgm.exe, which should be in
C:\mysql\bin
on the management node host. (Do not try to re-use the console window where ndb_mgmd.exe is running by typing CTRL+C, as this kills the management node.) The resulting output should look like this:C:\mysql\bin>
ndb_mgm
-- NDB Cluster -- Management Client -- ndb_mgm>When the prompt
ndb_mgm>
appears, this indicates that the management client is ready to receive MySQL Cluster management commands. You can observe the status of the data nodes as they start by enteringALL STATUS
at the management client prompt. This command causes a running report of the data nodes's startup sequence, which should look something like this:ndb_mgm>
ALL STATUS
Connected to Management Server at: localhost:1186 Node 2: starting (Last completed phase 3) (mysql-5.1.61-ndb-7.1.20) Node 3: starting (Last completed phase 3) (mysql-5.1.61-ndb-7.1.20) Node 2: starting (Last completed phase 4) (mysql-5.1.61-ndb-7.1.20) Node 3: starting (Last completed phase 4) (mysql-5.1.61-ndb-7.1.20) Node 2: Started (version 7.1.20) Node 3: Started (version 7.1.20) ndb_mgm>ЗамечаниеCommands issued in the management client are not case-sensitive; we use uppercase as the canonical form of these commands, but you are not required to observe this convention when inputting them into the ndb_mgm client. For more information, see Section 16.5.2, “Commands in the MySQL Cluster Management Client”.
The output produced by
ALL STATUS
is likely to vary from what is shown here, according to the speed at which the data nodes are able to start, the release version number of the MySQL Cluster software you are using, and other factors. What is significant is that, when you see that both data nodes have started, you are ready to start the SQL node.You can leave ndb_mgm.exe running; it has no negative impact on the performance of the MySQL Cluster, and we use it in the next step to verify that the SQL node is connected to the cluster after you have started it.
On the computer designated as the SQL node host, open a console window and navigate to the directory where you unpacked the MySQL Cluster binaries (if you are following our example, this is
C:\mysql\bin
).Start the SQL node by invoking mysqld.exe from the command line, as shown here:
C:\mysql\bin>
mysqld --console
The
--console
option causes logging information to be written to the console, which can be helpful in the event of problems. (Once you are satisfied that the SQL node is running in a satisfactory manner, you can stop it and restart it out without the--console
option, so that logging is performed normally.)In the console window where the management client (ndb_mgm.exe) is running on the management node host, enter the
SHOW
command, which should produce output similar to what is shown here:ndb_mgm>
SHOW
Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=2 @192.168.0.30 (Version: 5.1.61-ndb-7.1.20, Nodegroup: 0, Master) id=3 @192.168.0.40 (Version: 5.1.61-ndb-7.1.20, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.0.10 (Version: 5.1.61-ndb-7.1.20) [mysqld(API)] 1 node(s) id=4 @192.168.0.20 (Version: 5.1.61-ndb-7.1.20)You can also verify that the SQL node is connected to the MySQL Cluster in the mysql client (mysql.exe) using the
SHOW ENGINE NDB STATUS
statement.
You should now be ready to work with database objects and data
using MySQL Cluster's
NDBCLUSTER
storage engine. See
Section 16.2.5, “MySQL Cluster Пример with Tables and Data”, for more
information and examples.
You can also install ndb_mgmd.exe, ndbd.exe, and ndbmtd.exe as Windows services. For information on how to do this, see Section 16.2.2.4, “Installing MySQL Cluster Processes as Windows Services”).
Once you are satisfied that MySQL Cluster is running as desired, you can install the management nodes and data nodes as Windows services, so that these processes are started and stopped automatically whenever Windows is started or stopped. This also makes it possible to control these processes from the command line with the appropriate NET START or NET STOP command, or using the Windows graphical Services utility.
Installing programs as Windows services usually must be done using an account that has Administrator rights on the system.
To install the management node as a service on Windows, invoke
ndb_mgmd.exe from the command line on the
machine hosting the management node, using the
--install
option, as shown
here:
C:\> C:\mysql\bin\ndb_mgmd.exe --install
Installing service 'MySQL Cluster Management Server' as '"C:\mysql\bin\ndbd.exe" "--service=ndb_mgmd"'
Service successfully installed.
When installing a MySQL Cluster program as a Windows service, you should always specify the complete path; otherwise the service installation may fail with the error The system cannot find the file specified.
The --install
option must be
used first, ahead of any other options that might be specified
for ndb_mgmd.exe. However, it is preferable
to specify such options in an options file instead. If your
options file is not in one of the default locations as shown in
the output of ndb_mgmd.exe
--help
, you can specify the
location using the
--config-file
option.
Now you should be able to start and stop the management server like this:
C:\>NET START ndb_mgmd
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP ndb_mgmd
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
You can also start or stop the management server as a Windows service using the descriptive name, as shown here:
C:\>NET START 'MySQL Cluster Management Server'
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP 'MySQL Cluster Management Server'
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
However, it is usually simpler to specify a short service name
or to permit the default service name to be used when installing
the service, and then reference that name when starting or
stopping the service. To specify a service name other than
ndb_mgmd
, append it to the
--install
option, as shown in
this example:
C:\> C:\mysql\bin\ndb_mgmd.exe --install=mgmd1
Installing service 'MySQL Cluster Management Server' as '"C:\mysql\bin\ndb_mgmd.exe" "--service=mgmd1"'
Service successfully installed.
Now you should be able to start or stop the service using the name you have specified, like this:
C:\>NET START mgmd1
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP mgmd1
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
To remove the management node service, invoke
ndb_mgmd.exe with the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndb_mgmd.exe --remove
Removing service 'MySQL Cluster Management Server'
Service successfully removed.
If you installed the service using a service name other than the
default, you can remove the service by passing this name as the
value of the --remove
option,
like this:
C:\> C:\mysql\bin\ndb_mgmd.exe --remove=mgmd1
Removing service 'mgmd1'
Service successfully removed.
Installation of a MySQL Cluster data node processs as a Windows
service can be done in a similar fashion, using the
--install
option for
ndbd.exe (or ndbmtd.exe),
as shown here:
C:\> C:\mysql\bin\ndbd.exe --install
Installing service 'MySQL Cluster Data Node Daemon' as '"C:\mysql\bin\ndbd.exe" "--service=ndbd"'
Service successfully installed.
Now you can start or stop the data node using either the default service name or the descriptive name with net start or net stop, as shown in the following example:
C:\>NET START ndbd
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP ndbd
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully. C:\>NET START 'MySQL Cluster Data Node Daemon'
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP 'MySQL Cluster Data Node Daemon'
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully.
To remove the data node service, invoke
ndbd.exe with the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndbd.exe --remove
Removing service 'MySQL Cluster Data Node Daemon'
Service successfully removed.
As with ndb_mgmd.exe (and
mysqld.exe), when installing
ndbd.exe as a Windows service, you can also
specify a name for the service as the value of
--install
, and then use it when
starting or stopping the service, like this:
C:\>C:\mysql\bin\ndbd.exe --install=dnode1
Installing service 'dnode1' as '"C:\mysql\bin\ndbd.exe" "--service=dnode1"' Service successfully installed. C:\>NET START dnode1
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP dnode1
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully.
If you specified a service name when installing the data node
service, you can use this name when removing it as well, by
passing it as the value of the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndbd.exe --remove=dnode1
Removing service 'dnode1'
Service successfully removed.
Installation of the SQL node as a Windows service, starting the
service, stopping the service, and removing the service are done
in a similar fashion, using mysqld
--install
, NET START,
NET STOP, and mysqld
--remove
. For additional
information, see Section 2.3.6.7, “Starting MySQL as a Windows Service”.
For our four-node, four-host MySQL Cluster, it is necessary to write four configuration files, one per node host.
Each data node or SQL node requires a
my.cnf
file that provides two pieces of information: a connectstring that tells the node where to find the management node, and a line telling the MySQL server on this host (the machine hosting the data node) to enable theNDBCLUSTER
storage engine.For more information on connectstrings, see Section 16.3.2.3, “The MySQL Cluster Connectstring”.
The management node needs a
config.ini
file telling it how many replicas to maintain, how much memory to allocate for data and indexes on each data node, where to find the data nodes, where to save data to disk on each data node, and where to find any SQL nodes.
Configuring the data nodes and SQL nodes.
The my.cnf
file needed for the data nodes
is fairly simple. The configuration file should be located in
the /etc
directory and can be edited using
any text editor. (Create the file if it does not exist.) For
example:
shell> vi /etc/my.cnf
We show vi being used here to create the file, but any text editor should work just as well.
For each data node and SQL node in our example setup,
my.cnf
should look like this:
[mysqld] # Options for mysqld process: ndbcluster # run NDB storage engine ndb-connectstring=192.168.0.10 # location of management server [mysql_cluster] # Options for ndbd process: ndb-connectstring=192.168.0.10 # location of management server
After entering the preceding information, save this file and exit the text editor. Do this for the machines hosting data node “A”, data node “B”, and the SQL node.
Once you have started a mysqld process with
the NDBCLUSTER
and
ndb-connectstring
parameters in the
[mysqld]
in the my.cnf
file as shown previously, you cannot execute any
CREATE TABLE
or
ALTER TABLE
statements without
having actually started the cluster. Otherwise, these statements
will fail with an error. This is by design.
Configuring the management node.
The first step in configuring the management node is to create
the directory in which the configuration file can be found and
then to create the file itself. For example (running as
root
):
shell>mkdir /var/lib/mysql-cluster
shell>cd /var/lib/mysql-cluster
shell>vi config.ini
For our representative setup, the config.ini
file should read as follows:
[ndbd default] # Options affecting ndbd processes on all data nodes: NoOfReplicas=2 # Number of replicas DataMemory=80M # How much memory to allocate for data storage IndexMemory=18M # How much memory to allocate for index storage # For DataMemory and IndexMemory, we have used the # default values. Since the "world" database takes up # only about 500KB, this should be more than enough for # this example Cluster setup. [tcp default] # TCP/IP options: portnumber=2202 # This the default; however, you can use any # port that is free for all the hosts in the cluster # Note: It is recommended that you do not specify the port # number at all and simply allow the default value to be used # instead [ndb_mgmd] # Management process options: hostname=192.168.0.10 # Hostname or IP address of MGM node datadir=/var/lib/mysql-cluster # Directory for MGM node log files [ndbd] # Options for data node "A": # (one [ndbd] section per data node) hostname=192.168.0.30 # Hostname or IP address datadir=/usr/local/mysql/data # Directory for this data node's data files [ndbd] # Options for data node "B": hostname=192.168.0.40 # Hostname or IP address datadir=/usr/local/mysql/data # Directory for this data node's data files [mysqld] # SQL node options: hostname=192.168.0.20 # Hostname or IP address # (additional mysqld connections can be # specified for this node for various # purposes such as running ndb_restore)
The world
database can be downloaded from
http://dev.mysql.com/doc/, where it can be found listed
under “Examples”.
After all the configuration files have been created and these minimal options have been specified, you are ready to proceed with starting the cluster and verifying that all processes are running. We discuss how this is done in Section 16.2.4, “Initial Startup of MySQL Cluster”.
For more detailed information about the available MySQL Cluster configuration parameters and their uses, see Section 16.3.2, “MySQL Cluster Configuration Files”, and Section 16.3, “MySQL Cluster Configuration”. For configuration of MySQL Cluster as relates to making backups, see Section 16.5.3.3, “Configuration for MySQL Cluster Backups”.
The default port for Cluster management nodes is 1186; the default port for data nodes is 2202. However, the cluster can automatically allocate ports for data nodes from those that are already free.
Starting the cluster is not very difficult after it has been configured. Each cluster node process must be started separately, and on the host where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes:
On the management host, issue the following command from the system shell to start the management node process:
shell>
ndb_mgmd -f /var/lib/mysql-cluster/config.ini
The frist time that it is started, ndb_mgmd must be told where to find its configuration file, using the
-f
or--config-file
option. (See Section 16.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”, for details.)For additional options which can be used with ndb_mgmd, see Section 16.4.23, “Options Common to MySQL Cluster Programs”.
On each of the data node hosts, run this command to start the ndbd process:
shell>
ndbd
If you used RPM files to install MySQL on the cluster host where the SQL node is to reside, you can (and should) use the supplied startup script to start the MySQL server process on the SQL node.
If all has gone well, and the cluster has been set up correctly, the cluster should now be operational. You can test this by invoking the ndb_mgm management node client. The output should look like that shown here, although you might see some slight differences in the output depending upon the exact version of MySQL that you are using:
shell>ndb_mgm
-- NDB Cluster -- Management Client -- ndb_mgm>SHOW
Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=2 @192.168.0.30 (Version: 5.5.20-ndb-7.2.5, Nodegroup: 0, Master) id=3 @192.168.0.40 (Version: 5.5.20-ndb-7.2.5, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.0.10 (Version: 5.5.20-ndb-7.2.5) [mysqld(API)] 1 node(s) id=4 @192.168.0.20 (Version: 5.5.20-ndb-7.2.5)
The SQL node is referenced here as
[mysqld(API)]
, which reflects the fact that the
mysqld process is acting as a MySQL Cluster API
node.
The IP address shown for a given MySQL Cluster SQL or other API
node in the output of SHOW
is the address used by the SQL or API node to connect to the
cluster data nodes, and not to any management node.
You should now be ready to work with databases, tables, and data in MySQL Cluster. See Section 16.2.5, “MySQL Cluster Пример with Tables and Data”, for a brief discussion.
The information in this section applies to MySQL Cluster running on both Unix and Windows platforms.
Working with database tables and data in MySQL Cluster is not much different from doing so in standard MySQL. There are two key points to keep in mind:
For a table to be replicated in the cluster, it must use the
NDBCLUSTER
storage engine. To specify this, use theENGINE=NDBCLUSTER
orENGINE=NDB
option when creating the table:CREATE TABLE
tbl_name
(col_name
column_definitions
) ENGINE=NDBCLUSTER;Alternatively, for an existing table that uses a different storage engine, use
ALTER TABLE
to change the table to useNDBCLUSTER
:ALTER TABLE
tbl_name
ENGINE=NDBCLUSTER;Every
NDBCLUSTER
table has a primary key. If no primary key is defined by the user when a table is created, theNDBCLUSTER
storage engine automatically generates a hidden one. Such a key takes up space just as does any other table index. (It is not uncommon to encounter problems due to insufficient memory for accommodating these automatically created indexes.)
If you are importing tables from an existing database using the
output of mysqldump, you can open the SQL
script in a text editor and add the ENGINE
option to any table creation statements, or replace any existing
ENGINE
options. Suppose that you have the
world
sample database on another MySQL server
that does not support MySQL Cluster, and you want to export the
City
table:
shell> mysqldump --add-drop-table world City > city_table.sql
The resulting city_table.sql
file will
contain this table creation statement (and the
INSERT
statements necessary to
import the table data):
DROP TABLE IF EXISTS `City`;
CREATE TABLE `City` (
`ID` int(11) NOT NULL auto_increment,
`Name` char(35) NOT NULL default '',
`CountryCode` char(3) NOT NULL default '',
`District` char(20) NOT NULL default '',
`Population` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
INSERT INTO `City` VALUES (1,'Kabul','AFG','Kabol',1780000);
INSERT INTO `City` VALUES (2,'Qandahar','AFG','Qandahar',237500);
INSERT INTO `City` VALUES (3,'Herat','AFG','Herat',186800);(remaining INSERT statements omitted)
You need to make sure that MySQL uses the
NDBCLUSTER
storage engine for this
table. There are two ways that this can be accomplished. One of
these is to modify the table definition
before importing it into the Cluster
database. Using the City
table as an example,
modify the ENGINE
option of the definition as
follows:
DROP TABLE IF EXISTS `City`;
CREATE TABLE `City` (
`ID` int(11) NOT NULL auto_increment,
`Name` char(35) NOT NULL default '',
`CountryCode` char(3) NOT NULL default '',
`District` char(20) NOT NULL default '',
`Population` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) ENGINE=NDBCLUSTER DEFAULT CHARSET=latin1;
INSERT INTO `City` VALUES (1,'Kabul','AFG','Kabol',1780000);
INSERT INTO `City` VALUES (2,'Qandahar','AFG','Qandahar',237500);
INSERT INTO `City` VALUES (3,'Herat','AFG','Herat',186800);
(remaining INSERT statements omitted)
This must be done for the definition of each table that is to be
part of the clustered database. The easiest way to accomplish this
is to do a search-and-replace on the file that contains the
definitions and replace all instances of
TYPE=
or
engine_name
ENGINE=
with engine_name
ENGINE=NDBCLUSTER
. If you do not want to
modify the file, you can use the unmodified file to create the
tables, and then use ALTER TABLE
to
change their storage engine. The particulars are given later in
this section.
Assuming that you have already created a database named
world
on the SQL node of the cluster, you can
then use the mysql command-line client to read
city_table.sql
, and create and populate the
corresponding table in the usual manner:
shell> mysql world < city_table.sql
It is very important to keep in mind that the preceding command
must be executed on the host where the SQL node is running (in
this case, on the machine with the IP address
192.168.0.20
).
To create a copy of the entire world
database
on the SQL node, use mysqldump on the
noncluster server to export the database to a file named
world.sql
; for example, in the
/tmp
directory. Then modify the table
definitions as just described and import the file into the SQL
node of the cluster like this:
shell> mysql world < /tmp/world.sql
If you save the file to a different location, adjust the preceding instructions accordingly.
Running SELECT
queries on the SQL
node is no different from running them on any other instance of a
MySQL server. To run queries from the command line, you first need
to log in to the MySQL Monitor in the usual way (specify the
root
password at the Enter
password:
prompt):
shell> mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 5.5.20-ndb-7.2.5
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
We simply use the MySQL server's root
account and assume that you have followed the standard security
precautions for installing a MySQL server, including setting a
strong root
password. For more information, see
Section 2.10.2, “Securing the Initial MySQL Accounts”.
It is worth taking into account that Cluster nodes do
not make use of the MySQL privilege system
when accessing one another. Setting or changing MySQL user
accounts (including the root
account) effects
only applications that access the SQL node, not interaction
between nodes. See
Section 16.5.10.2, “MySQL Cluster and MySQL Privileges”, for
more information.
If you did not modify the ENGINE
clauses in the
table definitions prior to importing the SQL script, you should
run the following statements at this point:
mysql>USE world;
mysql>ALTER TABLE City ENGINE=NDBCLUSTER;
mysql>ALTER TABLE Country ENGINE=NDBCLUSTER;
mysql>ALTER TABLE CountryLanguage ENGINE=NDBCLUSTER;
Selecting a database and running a SELECT query against a table in that database is also accomplished in the usual manner, as is exiting the MySQL Monitor:
mysql>USE world;
mysql>SELECT Name, Population FROM City ORDER BY Population DESC LIMIT 5;
+-----------+------------+ | Name | Population | +-----------+------------+ | Bombay | 10500000 | | Seoul | 9981619 | | São Paulo | 9968485 | | Shanghai | 9696300 | | Jakarta | 9604900 | +-----------+------------+ 5 rows in set (0.34 sec) mysql>\q
Bye shell>
Applications that use MySQL can employ standard APIs to access
NDB
tables. It is important to
remember that your application must access the SQL node, and not
the management or data nodes. This brief example shows how we
might execute the SELECT
statement
just shown by using the PHP 5.X mysqli
extension running on a Web server elsewhere on the network:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<title>SIMPLE mysqli SELECT</title>
</head>
<body>
<?php
# connect to SQL node:
$link = new mysqli('192.168.0.20', 'root', 'root_password
', 'world');
# parameters for mysqli constructor are:
# host, user, password, database
if( mysqli_connect_errno() )
die("Connect failed: " . mysqli_connect_error());
$query = "SELECT Name, Population
FROM City
ORDER BY Population DESC
LIMIT 5";
# if no errors...
if( $result = $link->query($query) )
{
?>
<table border="1" width="40%" cellpadding="4" cellspacing ="1">
<tbody>
<tr>
<th width="10%">City</th>
<th>Population</th>
</tr>
<?
# then display the results...
while($row = $result->fetch_object())
printf("<tr>\n <td align=\"center\">%s</td><td>%d</td>\n</tr>\n",
$row->Name, $row->Population);
?>
</tbody
</table>
<?
# ...and verify the number of rows that were retrieved
printf("<p>Affected rows: %d</p>\n", $link->affected_rows);
}
else
# otherwise, tell us what went wrong
echo mysqli_error();
# free the result set and the mysqli connection object
$result->close();
$link->close();
?>
</body>
</html>
We assume that the process running on the Web server can reach the IP address of the SQL node.
In a similar fashion, you can use the MySQL C API, Perl-DBI, Python-mysql, or MySQL Connectors to perform the tasks of data definition and manipulation just as you would normally with MySQL.
To shut down the cluster, enter the following command in a shell on the machine hosting the management node:
shell> ndb_mgm -e shutdown
The -e
option here is used to pass a command to
the ndb_mgm client from the shell. (See
Section 16.4.23, “Options Common to MySQL Cluster Programs”, for more
information about this option.) The command causes the
ndb_mgm, ndb_mgmd, and any
ndbd or ndbmtd processes to
terminate gracefully. Any SQL nodes can be terminated using
mysqladmin shutdown and other means. On Windows
platforms, assuming that you have installed the SQL node as a
Windows service, you can use NET STOP MYSQL.
To restart the cluster on Unix platforms, run these commands:
On the management host (
192.168.0.10
in our example setup):shell>
ndb_mgmd -f /var/lib/mysql-cluster/config.ini
On each of the data node hosts (
192.168.0.30
and192.168.0.40
):shell>
ndbd
Use the ndb_mgm client to verify that both data nodes have started successfully.
On the SQL host (
192.168.0.20
):shell>
mysqld_safe &
On Windows platforms, assuming that you have installed all MySQL Cluster processes as Windows services using the default service names (see Section 16.2.2.4, “Installing MySQL Cluster Processes as Windows Services”), you can restart the cluster as follows:
On the management host (
192.168.0.10
in our example setup), execute the following command:C:\>
NET START ndb_mgmd
On each of the data node hosts (
192.168.0.30
and192.168.0.40
), execute the following command:C:\>
NET START ndbd
On the management node host, use the ndb_mgm client to verify that the management node and both data nodes have started successfully (see Section 16.2.2.3, “Initial Startup of MySQL Cluster on Windows”).
On the SQL node host (
192.168.0.20
), execute the following command:C:\>
NET START mysql
In a production setting, it is usually not desirable to shut down the cluster completely. In many cases, even when making configuration changes, or performing upgrades to the cluster hardware or software (or both), which require shutting down individual host machines, it is possible to do so without shutting down the cluster as a whole by performing a rolling restart of the cluster. For more information about doing this, see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”.
This section provides information about MySQL Cluster software and table file compatibility between different MySQL Cluster NDB 7.2 releases with regard to performing upgrades and downgrades as well as compatibility matrices and notes. You are expected already to be familiar with installing and configuring a MySQL Cluster prior to attempting an upgrade or downgrade. See Section 16.3, “MySQL Cluster Configuration”.
Only compatibility between MySQL versions with regard to
NDBCLUSTER
is taken into account in
this section, and there are likely other issues to be
considered. As with any other MySQL software upgrade
or downgrade, you are strongly encouraged to review the relevant
portions of the MySQL Manual for the MySQL versions from which
and to which you intend to migrate, before attempting an upgrade
or downgrade of the MySQL Cluster software. This is
especially true when planning a migration from MySQL Cluster NDB
7.1 (or earlier) to MySQL Cluster NDB 7.2, since the version of
the underlying MySQL Server also changes from MySQL 5.1 to MySQL
5.5. See Section 2.11.1, “Upgrading MySQL”.
Versions supported. The following versions of MySQL Cluster are supported for upgrades to MySQL Cluster NDB 7.2 (7.2.4 and later):
MySQL Cluster NDB 7.1 GA releases (7.1.3 and later)
MySQL Cluster NDB 7.0 GA releases (7.0.5 and later)
MySQL Cluster NDB 6.3 GA releases (6.3.8 and later) that can be upgraded to MySQL Cluster NDB 7.1
For information about upgrades and downgrades in previous MySQL Cluster release series, see Upgrade and Downgrade Compatibility: MySQL Cluster NDB 6.x, and Upgrade and downgrade compatibility: MySQL Cluster NDB 7.x.
NDB API, ClusterJ, and other appplications used with recent releases of MySQL Cluster NDB 6.3 and later should continue to work with MySQL Cluster NDB 7.2.4 and later without rewriting or recompiling.
Upgrading ndbd to ndbmtd. When upgrading online from a MySQL Cluster NDB 6.3 release to a MySQL Cluster NDB 7.0 or later release, you should not try to upgrade the data nodes from ndbd to ndbmtd at the same time. Instead, perform the upgrade using the new ndbd executable (from the MySQL Cluster distribution to which you are upgrading) to replace the one in use on the data nodes. Once the version upgrade is complete, you can perform a second (online) upgrade to replace the data node executables with ndbmtd from the “new” MySQL Cluster distribution.
A MySQL server that is part of a MySQL Cluster differs in one chief
respect from a normal (nonclustered) MySQL server, in that it
employs the NDBCLUSTER
storage engine.
This engine is also referred to simply as
NDB
, and the two forms of the name are
synonymous.
To avoid unnecessary allocation of resources, the server is
configured by default with the NDB
storage engine disabled. To enable NDB
,
you must modify the server's my.cnf
configuration file, or start the server with the
--ndbcluster
option.
For more information about
--ndbcluster
and other MySQL server
options specific to MySQL Cluster, see
Section 16.3.4.2, “MySQL Server Options for MySQL Cluster”.
The MySQL server is a part of the cluster, so it also must know how
to access an MGM node to obtain the cluster configuration data. The
default behavior is to look for the MGM node on
localhost
. However, should you need to specify
that its location is elsewhere, this can be done in
my.cnf
or on the MySQL server command line.
Before the NDB
storage engine can be
used, at least one MGM node must be operational, as well as any
desired data nodes.
NDB
, the MySQL Cluster storage engine,
is available in binary distributions for Linux, Mac OS X, Solaris.
and Windows. We are working to support MySQL Cluster on all
operating systems supported by the MySQL Server. For information
about installing MySQL Cluster, see
Section 16.2, “MySQL Cluster Installation”.
To familiarize you with the basics, we will describe the simplest possible configuration for a functional MySQL Cluster. After this, you should be able to design your desired setup from the information provided in the other relevant sections of this chapter.
First, you need to create a configuration directory such as
/var/lib/mysql-cluster
, by executing the
following command as the system root
user:
shell> mkdir /var/lib/mysql-cluster
In this directory, create a file named
config.ini
that contains the following
information. Substitute appropriate values for
HostName
and DataDir
as
necessary for your system.
# file "config.ini" - showing minimal setup consisting of 1 data node, # 1 management server, and 3 MySQL servers. # The empty default sections are not required, and are shown only for # the sake of completeness. # Data nodes must provide a hostname but MySQL Servers are not required # to do so. # If you don't know the hostname for your machine, use localhost. # The DataDir parameter also has a default value, but it is recommended to # set it explicitly. # Note: [db], [api], and [mgm] are aliases for [ndbd], [mysqld], and [ndb_mgmd], # respectively. [db] is deprecated and should not be used in new installations. [ndbd default] NoOfReplicas= 1 [mysqld default] [ndb_mgmd default] [tcp default] [ndb_mgmd] HostName= myhost.example.com [ndbd] HostName= myhost.example.com DataDir= /var/lib/mysql-cluster [mysqld] [mysqld] [mysqld]
You can now start the ndb_mgmd management
server. By default, it attempts to read the
config.ini
file in its current working
directory, so change location into the directory where the file is
located and then invoke ndb_mgmd:
shell>cd /var/lib/mysql-cluster
shell>ndb_mgmd
Then start a single data node by running ndbd:
shell> ndbd
For command-line options which can be used when starting ndbd, see Section 16.4.23, “Options Common to MySQL Cluster Programs”.
By default, ndbd looks for the management
server at localhost
on port 1186.
If you have installed MySQL from a binary tarball, you will need
to specify the path of the ndb_mgmd and
ndbd servers explicitly. (Normally, these
will be found in /usr/local/mysql/bin
.)
Finally, change location to the MySQL data directory (usually
/var/lib/mysql
or
/usr/local/mysql/data
), and make sure that
the my.cnf
file contains the option necessary
to enable the NDB storage engine:
[mysqld] ndbcluster
You can now start the MySQL server as usual:
shell> mysqld_safe --user=mysql &
Wait a moment to make sure the MySQL server is running properly.
If you see the notice mysql ended
, check the
server's .err
file to find out what went
wrong.
If all has gone well so far, you now can start using the cluster.
Connect to the server and verify that the
NDBCLUSTER
storage engine is enabled:
shell>mysql
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 to server version: 5.5.22 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>SHOW ENGINES\G
... *************************** 12. row *************************** Engine: NDBCLUSTER Support: YES Comment: Clustered, fault-tolerant, memory-based tables *************************** 13. row *************************** Engine: NDB Support: YES Comment: Alias for NDBCLUSTER ...
The row numbers shown in the preceding example output may be different from those shown on your system, depending upon how your server is configured.
Try to create an NDBCLUSTER
table:
shell>mysql
mysql>USE test;
Database changed mysql>CREATE TABLE ctest (i INT) ENGINE=NDBCLUSTER;
Query OK, 0 rows affected (0.09 sec) mysql>SHOW CREATE TABLE ctest \G
*************************** 1. row *************************** Table: ctest Create Table: CREATE TABLE `ctest` ( `i` int(11) default NULL ) ENGINE=ndbcluster DEFAULT CHARSET=latin1 1 row in set (0.00 sec)
To check that your nodes were set up properly, start the management client:
shell> ndb_mgm
Use the SHOW command from within the management client to obtain a report on the cluster's status:
ndb_mgm> SHOW
Cluster Configuration
---------------------
[ndbd(NDB)] 1 node(s)
id=2 @127.0.0.1 (Version: 5.5.20-ndb-7.2.5, Nodegroup: 0, Master)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @127.0.0.1 (Version: 5.5.20-ndb-7.2.5)
[mysqld(API)] 3 node(s)
id=3 @127.0.0.1 (Version: 5.5.20-ndb-7.2.5)
id=4 (not connected, accepting connect from any host)
id=5 (not connected, accepting connect from any host)
At this point, you have successfully set up a working MySQL
Cluster. You can now store data in the cluster by using any table
created with ENGINE=NDBCLUSTER
or its alias
ENGINE=NDB
.
- 16.3.2.1. MySQL Cluster Configuration: Basic Пример
- 16.3.2.2. Recommended Starting Configuration for MySQL Cluster
- 16.3.2.3. The MySQL Cluster Connectstring
- 16.3.2.4. Defining Computers in a MySQL Cluster
- 16.3.2.5. Defining a MySQL Cluster Management Server
- 16.3.2.6. Defining MySQL Cluster Data Nodes
- 16.3.2.7. Defining SQL and Other API Nodes in a MySQL Cluster
- 16.3.2.8. MySQL Cluster TCP/IP Connections
- 16.3.2.9. MySQL Cluster TCP/IP Connections Using Direct Connections
- 16.3.2.10. MySQL Cluster Shared-Memory Connections
- 16.3.2.11. SCI Transport Connections in MySQL Cluster
- 16.3.2.12. Configuring MySQL Cluster Send Buffer Parameters
Configuring MySQL Cluster requires working with two files:
my.cnf
: Specifies options for all MySQL Cluster executables. This file, with which you should be familiar with from previous work with MySQL, must be accessible by each executable running in the cluster.config.ini
: This file, sometimes known as the global configuration file, is read only by the MySQL Cluster management server, which then distributes the information contained therein to all processes participating in the cluster.config.ini
contains a description of each node involved in the cluster. This includes configuration parameters for data nodes and configuration parameters for connections between all nodes in the cluster. For a quick reference to the sections that can appear in this file, and what sorts of configuration parameters may be placed in each section, see Sections of theconfig.ini
File.
Caching of configuration data. In MySQL Cluster NDB 7.2, MySQL Cluster uses stateful configuration. Rather than reading the global configuration file every time the management server is restarted, the management server caches the configuration the first time it is started, and thereafter, the global configuration file is read only when one of the following conditions is true:
The management server is started using the
--initial
option. In this case, the global configuration file is re-read, any existing cache files are deleted, and the management server creates a new configuration cache.The management server is started using the
--reload
option. In this case, the management server compares its cache with the global configuration file. If they differ, the management server creates a new configuration cache; any existing configuration cache is preserved, but not used. If the management server's cache and the global configuration file contain the same configuration data, then the existing cache is used, and no new cache is created.The management server is started using a
--config-cache
option. This option can be used to force the management server to bypass configuration caching altogether. In this case, the management server ignores any configuration files that may be present, always reading its configuration data from theconfig.ini
file instead.No configuration cache is found. In this case, the management server reads the global configuration file and creates a cache containing the same configuration data as found in the file.
Configuration cache files.
The management server by default creates configuration cache
files in a directory named mysql-cluster
in
the MySQL installation directory. (If you build MySQL Cluster
from source on a Unix system, the default location is
/usr/local/mysql-cluster
.) This can be
overridden at run time by starting the management server with
the --configdir
option. Configuration cache
files are binary files named according to the pattern
ndb_
,
where node_id
_config.bin.seq_id
node_id
is the management
server's node ID in the cluster, and
seq_id
is a cache idenitifer. Cache
files are numbered sequentially using
seq_id
, in the order in which they
are created. The management server uses the latest cache file as
determined by the seq_id
.
It is possible to roll back to a previous configuration by
deleting later configuration cache files, or by renaming an
earlier cache file so that it has a higher
seq_id
. However, since configuration
cache files are written in a binary format, you should not
attempt to edit their contents by hand.
For more information about the --configdir
,
--initial
, and --reload
options
for the MySQL Cluster management server, see
Section 16.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.
We are continuously making improvements in Cluster configuration and attempting to simplify this process. Although we strive to maintain backward compatibility, there may be times when introduce an incompatible change. In such cases we will try to let Cluster users know in advance if a change is not backward compatible. If you find such a change and we have not documented it, please report it in the MySQL bugs database using the instructions given in Section 1.7, “How to Report Bugs or Problems”.
To support MySQL Cluster, you will need to update
my.cnf
as shown in the following example.
You may also specify these parameters on the command line when
invoking the executables.
The options shown here should not be confused with those that
are used in config.ini
global
configuration files. Global configuration options are
discussed later in this section.
# my.cnf # example additions to my.cnf for MySQL Cluster # (valid in MySQL 5.5) # enable ndbcluster storage engine, and provide connectstring for # management server host (default port is 1186) [mysqld] ndbcluster ndb-connectstring=ndb_mgmd.mysql.com # provide connectstring for management server host (default port: 1186) [ndbd] connect-string=ndb_mgmd.mysql.com # provide connectstring for management server host (default port: 1186) [ndb_mgm] connect-string=ndb_mgmd.mysql.com # provide location of cluster configuration file [ndb_mgmd] config-file=/etc/config.ini
(For more information on connectstrings, see Section 16.3.2.3, “The MySQL Cluster Connectstring”.)
# my.cnf # example additions to my.cnf for MySQL Cluster # (will work on all versions) # enable ndbcluster storage engine, and provide connectstring for management # server host to the default port 1186 [mysqld] ndbcluster ndb-connectstring=ndb_mgmd.mysql.com:1186
Once you have started a mysqld process with
the NDBCLUSTER
and
ndb-connectstring
parameters in the
[mysqld]
in the my.cnf
file as shown previously, you cannot execute any
CREATE TABLE
or
ALTER TABLE
statements without
having actually started the cluster. Otherwise, these
statements will fail with an error. This is by
design.
You may also use a separate [mysql_cluster]
section in the cluster my.cnf
file for
settings to be read and used by all executables:
# cluster-specific settings [mysql_cluster] ndb-connectstring=ndb_mgmd.mysql.com:1186
For additional NDB
variables that
can be set in the my.cnf
file, see
Section 16.3.4.3, “MySQL Cluster System Variables”.
The MySQL Cluster global configuration file is named
config.ini
by default. It is read by
ndb_mgmd at startup and can be placed
anywhere. Its location and name are specified by using
--config-file=
on the ndb_mgmd command line. If the
configuration file is not specified, ndb_mgmd
by default tries to read a file named
path_name
config.ini
located in the current working
directory.
The global configuration file for MySQL Cluster uses INI format,
which consists of sections preceded by section headings
(surrounded by square brackets), followed by the appropriate
parameter names and values. One deviation from the standard INI
format is that the parameter name and value can be separated by
a colon (“:
”) as well as the
equal sign (“=
”); however, the
equal sign is preferred. Another deviation is that sections are
not uniquely identified by section name. Instead, unique
sections (such as two different nodes of the same type) are
identified by a unique ID specified as a parameter within the
section.
Default values are defined for most parameters, and can also be
specified in config.ini
. To create a
default value section, simply add the word
default
to the section name. For example, an
[ndbd]
section contains parameters that apply
to a particular data node, whereas an [ndbd
default]
section contains parameters that apply to all
data nodes. Suppose that all data nodes should use the same data
memory size. To configure them all, create an [ndbd
default]
section that contains a
DataMemory
line to
specify the data memory size.
In some older releases of MySQL Cluster, there was no default
value for
NoOfReplicas
, which
always had to be specified explicitly in the [ndbd
default]
section. Although this parameter now has a
default value of 2, which is the recommended setting in most
common usage scenarios, it is still recommended practice to
set this parameter explicitly.
The global configuration file must define the computers and nodes involved in the cluster and on which computers these nodes are located. An example of a simple configuration file for a cluster consisting of one management server, two data nodes and two MySQL servers is shown here:
# file "config.ini" - 2 data nodes and 2 SQL nodes # This file is placed in the startup directory of ndb_mgmd (the # management server) # The first MySQL Server can be started from any host. The second # can be started only on the host mysqld_5.mysql.com [ndbd default] NoOfReplicas= 2 DataDir= /var/lib/mysql-cluster [ndb_mgmd] Hostname= ndb_mgmd.mysql.com DataDir= /var/lib/mysql-cluster [ndbd] HostName= ndbd_2.mysql.com [ndbd] HostName= ndbd_3.mysql.com [mysqld] [mysqld] HostName= mysqld_5.mysql.com
The preceding example is intended as a minimal starting configuration for purposes of familiarization with MySQL Cluster, and is almost certain not to be sufficient for production settings. See Section 16.3.2.2, “Recommended Starting Configuration for MySQL Cluster”, which provides a more complete example starting configuration.
Each node has its own section in the
config.ini
file. For example, this cluster
has two data nodes, so the preceding configuration file contains
two [ndbd]
sections defining these nodes.
Do not place comments on the same line as a section heading in
the config.ini
file; this causes the
management server not to start because it cannot parse the
configuration file in such cases.
Sections of the config.ini
File
There are six different sections that you can use in the
config.ini
configuration file, as described
in the following list:
[computer]
: Defines cluster hosts. This is not required to configure a viable MySQL Cluster, but be may used as a convenience when setting up a large cluster. See Section 16.3.2.4, “Defining Computers in a MySQL Cluster”, for more information.[ndbd]
: Defines a cluster data node (ndbd process). See Section 16.3.2.6, “Defining MySQL Cluster Data Nodes”, for details.[mysqld]
: Defines the cluster's MySQL server nodes (also called SQL or API nodes). For a discussion of SQL node configuration, see Section 16.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.[mgm]
or[ndb_mgmd]
: Defines a cluster management server (MGM) node. For information concerning the configuration of MGM nodes, see Section 16.3.2.5, “Defining a MySQL Cluster Management Server”.[tcp]
: Defines a TCP/IP connection between cluster nodes, with TCP/IP being the default connection protocol. Normally,[tcp]
or[tcp default]
sections are not required to set up a MySQL Cluster, as the cluster handles this automatically; however, it may be necessary in some situations to override the defaults provided by the cluster. See Section 16.3.2.8, “MySQL Cluster TCP/IP Connections”, for information about available TCP/IP configuration parameters and how to use them. (You may also find Section 16.3.2.9, “MySQL Cluster TCP/IP Connections Using Direct Connections” to be of interest in some cases.)[shm]
: Defines shared-memory connections between nodes. In MySQL 5.5, it is enabled by default, but should still be considered experimental. For a discussion of SHM interconnects, see Section 16.3.2.10, “MySQL Cluster Shared-Memory Connections”.[sci]
:Defines Scalable Coherent Interface connections between cluster data nodes. Such connections require software which, while freely available, is not part of the MySQL Cluster distribution, as well as specialized hardware. See Section 16.3.2.11, “SCI Transport Connections in MySQL Cluster” for detailed information about SCI interconnects.
You can define default
values for each
section. All Cluster parameter names are case-insensitive, which
differs from parameters specified in my.cnf
or my.ini
files.
Achieving the best performance from a MySQL Cluster depends on a number of factors including the following:
MySQL Cluster software version
Numbers of data nodes and SQL nodes
Hardware
Operating system
Amount of data to be stored
Size and type of load under which the cluster is to operate
Therefore, obtaining an optimum configuration is likely to be an iterative process, the outcome of which can vary widely with the specifics of each MySQL Cluster deployment. Changes in configuration are also likely to be indicated when changes are made in the platform on which the cluster is run, or in applications that use the MySQL Cluster's data. For these reasons, it is not possible to offer a single configuration that is ideal for all usage scenarios. However, in this section, we provide a recommended base configuration.
Starting config.ini
file.
The following config.ini
file is a
recommended starting point for configuring a cluster running
MySQL Cluster NDB 7.2:
# TCP PARAMETERS [tcp default]SendBufferMemory=2M ReceiveBufferMemory=2M # Increasing the sizes of these 2 buffers beyond the default values # helps prevent bottlenecks due to slow disk I/O. # MANAGEMENT NODE PARAMETERS [ndb_mgmd default] DataDir=path/to/management/server/data/directory
# It is possible to use a different data directory for each management # server, but for ease of administration it is preferable to be # consistent. [ndb_mgmd] HostName=management-server-A-hostname
# NodeId=management-server-A-nodeid
[ndb_mgmd] HostName=management-server-B-hostname
# NodeId=management-server-B-nodeid
# Using 2 management servers helps guarantee that there is always an # arbitrator in the event of network partitioning, and so is # recommended for high availability. Each management server must be # identified by a HostName. You may for the sake of convenience specify # a NodeId for any management server, although one will be allocated # for it automatically; if you do so, it must be in the range 1-255 # inclusive and must be unique among all IDs specified for cluster # nodes. # DATA NODE PARAMETERS [ndbd default] NoOfReplicas=2 # Using 2 replicas is recommended to guarantee availability of data; # using only 1 replica does not provide any redundancy, which means # that the failure of a single data node causes the entire cluster to # shut down. We do not recommend using more than 2 replicas, since 2 is # sufficient to provide high availability, and we do not currently test # with greater values for this parameter. LockPagesInMainMemory=1 # On Linux and Solaris systems, setting this parameter locks data node # processes into memory. Doing so prevents them from swapping to disk, # which can severely degrade cluster performance. DataMemory=3072M IndexMemory=384M # The values provided for DataMemory and IndexMemory assume 4 GB RAM # per data node. However, for best results, you should first calculate # the memory that would be used based on the data you actually plan to # store (you may find the ndb_size.pl utility helpful in estimating # this), then allow an extra 20% over the calculated values. Naturally, # you should ensure that each data node host has at least as much # physical memory as the sum of these two values. # ODirect=1 # Enabling this parameter causes NDBCLUSTER to try using O_DIRECT # writes for local checkpoints and redo logs; this can reduce load on # CPUs. We recommend doing so when using MySQL Cluster on systems running # Linux kernel 2.6 or later. NoOfFragmentLogFiles=300 DataDir=path/to/data/node/data/directory
MaxNoOfConcurrentOperations=100000 SchedulerSpinTimer=400 SchedulerExecutionTimer=100 RealTimeScheduler=1 # Setting these parameters allows you to take advantage of real-time scheduling # of NDBCLUSTER threads to get higher throughput. TimeBetweenGlobalCheckpoints=1000 TimeBetweenEpochs=200 DiskCheckpointSpeed=10M DiskCheckpointSpeedInRestart=100M RedoBuffer=32M # CompressedLCP=1 # CompressedBackup=1 # Enabling CompressedLCP and CompressedBackup causes, respectively, local checkpoint files and backup files to be compressed, which can result in a space savings of up to 50% over noncompressed LCPs and backups. # MaxNoOfLocalScans=64 MaxNoOfTables=1024 MaxNofOfOrderedIndexes=256 [ndbd] HostName=data-node-A-hostname
# NodeId=data-node-A-nodeid
LockExecuteThreadToCPU=1 LockMaintThreadsToCPU=0 # On systems with multiple CPUs, these parameters can be used to lock NDBCLUSTER # threads to specific CPUs [ndbd] HostName=data-node-B-hostname
# NodeId=data-node-B-nodeid
LockExecuteThreadToCPU=1 LockMaintThreadsToCPU=0 # You must have an [ndbd] section for every data node in the cluster; # each of these sections must include a HostName. Each section may # optionally include a NodeId for convenience, but in most cases, it is # sufficient to allow the cluster to allocate node IDs dynamically. If # you do specify the node ID for a data node, it must be in the range 1 # to 48 inclusive and must be unique among all IDs specified for # cluster nodes. # SQL NODE / API NODE PARAMETERS [mysqld] # HostName=sql-node-A-hostname
# NodeId=sql-node-A-nodeid
[mysqld] [mysqld] # Each API or SQL node that connects to the cluster requires a [mysqld] # or [api] section of its own. Each such section defines a connection # “slot”; you should have at least as many of these sections in the # config.ini file as the total number of API nodes and SQL nodes that # you wish to have connected to the cluster at any given time. There is # no performance or other penalty for having extra slots available in # case you find later that you want or need more API or SQL nodes to # connect to the cluster at the same time. # If no HostName is specified for a given [mysqld] or [api] section, # then any API or SQL node may use that slot to connect to the # cluster. You may wish to use an explicit HostName for one connection slot # to guarantee that an API or SQL node from that host can always # connect to the cluster. If you wish to prevent API or SQL nodes from # connecting from other than a desired host or hosts, then use a # HostName for every [mysqld] or [api] section in the config.ini file. # You can if you wish define a node ID (NodeId parameter) for any API or # SQL node, but this is not necessary; if you do so, it must be in the # range 1 to 255 inclusive and must be unique among all IDs specified # for cluster nodes.
Recommended my.cnf
options for SQL nodes.
MySQL Servers acting as MySQL Cluster SQL nodes must always be
started with the --ndbcluster
and --ndb-connectstring
options, either on
the command line or in my.cnf
. In
addition, set the following options for all
mysqld processes in the cluster, unless
your setup requires otherwise:
--ndb-use-exact-count=0
--ndb-index-stat-enable=0
--ndb-force-send=1
--engine-condition-pushdown=1
With the exception of the MySQL Cluster management server (ndb_mgmd), each node that is part of a MySQL Cluster requires a connectstring that points to the management server's location. This connectstring is used in establishing a connection to the management server as well as in performing other tasks depending on the node's role in the cluster. The syntax for a connectstring is as follows:
[nodeid=node_id
, ]host-definition
[,host-definition
[, ...]]host-definition
:host_name
[:port_number
]
node_id
is an integer larger than 1 which
identifies a node in config.ini
.
host_name
is a string representing a
valid Internet host name or IP address.
port_number
is an integer referring
to a TCP/IP port number.
example 1 (long): "nodeid=2,myhost1:1100,myhost2:1100,192.168.0.3:1200" example 2 (short): "myhost1"
localhost:1186
is used as the default
connectstring value if none is provided. If
port_num
is omitted from the
connectstring, the default port is 1186. This port should always
be available on the network because it has been assigned by IANA
for this purpose (see
http://www.iana.org/assignments/port-numbers for
details).
By listing multiple host definitions, it is possible to designate several redundant management servers. A MySQL Cluster data or API node attempts to contact successive management servers on each host in the order specified, until a successful connection has been established.
It is also possible to specify in a connectstring one or more bind addresses to be used by nodes having multiple network interfaces for connecting to management servers. A bind address consists of a hostname or network address and an optional port number. This enhanced syntax for connectstrings is shown here:
[nodeid=node_id
, ] [bind-address=host-definition
, ]host-definition
[; bind-address=host-definition
]host-definition
[; bind-address=host-definition
] [, ...]]host-definition
:host_name
[:port_number
]
If a single bind address is used in the connectstring
prior to specifying any management hosts,
then this address is used as the default for connecting to any
of them (unless overridden for a given management server; see
later in this section for an example). For example, the
following connectstring causes the node to use
192.168.178.242
regardless of the management
server to which it connects:
bind-address=192.168.178.242, poseidon:1186, perch:1186
If a bind address is specified following a management host definition, then it is used only for connecting to that management node. Consider the following connectstring:
poseidon:1186;bind-address=localhost, perch:1186;bind-address=192.168.178.242
In this case, the node uses localhost
to
connect to the management server running on the host named
poseidon
and
192.168.178.242
to connect to the management
server running on the host named perch
.
You can specify a default bind address and then override this
default for one or more specific management hosts. In the
following example, localhost
is used for
connecting to the management server running on host
poseidon
; since
192.168.178.242
is specified first (before
any management server definitions), it is the default bind
address and so is used for connecting to the management servers
on hosts perch
and orca
:
bind-address=192.168.178.242,poseidon:1186;bind-address=localhost,perch:1186,orca:2200
There are a number of different ways to specify the connectstring:
Each executable has its own command-line option which enables specifying the management server at startup. (See the documentation for the respective executable.)
It is also possible to set the connectstring for all nodes in the cluster at once by placing it in a
[mysql_cluster]
section in the management server'smy.cnf
file.For backward compatibility, two other options are available, using the same syntax:
Set the
NDB_CONNECTSTRING
environment variable to contain the connectstring.Write the connectstring for each executable into a text file named
Ndb.cfg
and place this file in the executable's startup directory.
However, these are now deprecated and should not be used for new installations.
The recommended method for specifying the connectstring is to
set it on the command line or in the my.cnf
file for each executable.
The [computer]
section has no real
significance other than serving as a way to avoid the need of
defining host names for each node in the system. All parameters
mentioned here are required.
Restart Type initial, node Permitted Values Type string
Default Range ..
This is a unique identifier, used to refer to the host computer elsewhere in the configuration file.
ImportantThe computer ID is not the same as the node ID used for a management, API, or data node. Unlike the case with node IDs, you cannot use
NodeId
in place ofId
in the[computer]
section of theconfig.ini
file.Restart Type system Permitted Values Type string
Default Range ..
This is the computer's hostname or IP address.
The [ndb_mgmd]
section is used to configure
the behavior of the management server. [mgm]
can be used as an alias; the two section names are equivalent.
All parameters in the following list are optional and assume
their default values if omitted.
If neither the ExecuteOnComputer
nor the
HostName
parameter is present, the default
value localhost
will be assumed for both.
Deprecated 5.1.51-ndb-7.1.9 Restart Type node Permitted Values Type numeric
Default Range 1 .. 63
Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 255, inclusive. This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.
ЗамечаниеData node IDs must be less than 49. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for management nodes (and API nodes) to values greater than 48.
The use of the
Id
parameter for identifying management nodes is deprecated in favor ofNodeId
. AlthoughId
continues to be supported for backward compatibility, it now generates a warning and is subject to removal in a future version of MySQL Cluster.Restart Type node Permitted Values Type numeric
Default Range 1 .. 63
Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 255 inclusive. This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.
ЗамечаниеData node IDs must be less than 49. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for management nodes (and API nodes) to values greater than 48.
NodeId
is the preferred parameter name to use when identifying management nodes. Although the olderId
continues to be supported for backward compatibility, it is now deprecated and generates a warning when used; it is also subject to removal in a future MySQL Cluster release.Restart Type system Permitted Values Type string
Default Range ..
This refers to the
Id
set for one of the computers defined in a[computer]
section of theconfig.ini
file.Restart Type node Permitted Values Type numeric
Default 1186
Range 0 .. 64K
This is the port number on which the management server listens for configuration requests and management commands.
Restart Type system Permitted Values Type string
Default Range ..
Specifying this parameter defines the hostname of the computer on which the management node is to reside. To specify a hostname other than
localhost
, either this parameter orExecuteOnComputer
is required.Restart Type node Permitted Values Type string
Default FILE:filename=ndb_nodeid_cluster.log,maxsize=1000000,maxfiles=6
Range ..
This parameter specifies where to send cluster logging information. There are three options in this regard—
CONSOLE
,SYSLOG
, andFILE
—withFILE
being the default:CONSOLE
outputs the log tostdout
:CONSOLE
SYSLOG
sends the log to asyslog
facility, possible values being one ofauth
,authpriv
,cron
,daemon
,ftp
,kern
,lpr
,mail
,news
,syslog
,user
,uucp
,local0
,local1
,local2
,local3
,local4
,local5
,local6
, orlocal7
.ЗамечаниеNot every facility is necessarily supported by every operating system.
SYSLOG:facility=syslog
FILE
pipes the cluster log output to a regular file on the same machine. The following values can be specified:filename
: The name of the log file.maxsize
: The maximum size (in bytes) to which the file can grow before logging rolls over to a new file. When this occurs, the old log file is renamed by appending.N
to the file name, whereN
is the next number not yet used with this name.maxfiles
: The maximum number of log files.
FILE:filename=cluster.log,maxsize=1000000,maxfiles=6
The default value for the
FILE
parameter isFILE:filename=ndb_
, wherenode_id
_cluster.log,maxsize=1000000,maxfiles=6node_id
is the ID of the node.
It is possible to specify multiple log destinations separated by semicolons as shown here:
CONSOLE;SYSLOG:facility=local0;FILE:filename=/var/log/mgmd
Restart Type node Permitted Values Type numeric
Default 1
Range 0 .. 2
This parameter is used to define which nodes can act as arbitrators. Only management nodes and SQL nodes can be arbitrators.
ArbitrationRank
can take one of the following values:0
: The node will never be used as an arbitrator.1
: The node has high priority; that is, it will be preferred as an arbitrator over low-priority nodes.2
: Indicates a low-priority node which be used as an arbitrator only if a node with a higher priority is not available for that purpose.
Normally, the management server should be configured as an arbitrator by setting its
ArbitrationRank
to 1 (the default for management nodes) and those for all SQL nodes to 0 (the default for SQL nodes).You can disable arbitration completely either by setting
ArbitrationRank
to 0 on all management and SQL nodes, or by setting theArbitration
parameter in the[ndbd default]
section of theconfig.ini
global configuration file. SettingArbitration
causes any settings forArbitrationRank
to be disregarded.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
An integer value which causes the management server's responses to arbitration requests to be delayed by that number of milliseconds. By default, this value is 0; it is normally not necessary to change it.
Restart Type node Permitted Values Type string
Default .
Range ..
This specifies the directory where output files from the management server will be placed. These files include cluster log files, process output files, and the daemon's process ID (PID) file. (For log files, this location can be overridden by setting the
FILE
parameter forLogDestination
as discussed previously in this section.)The default value for this parameter is the directory in which ndb_mgmd is located.
Set the scheduling policy and priority of heartbeat threads for management and API nodes.
The syntax for setting this parameter is shown here:
HeartbeatThreadPriority =
policy
[,priority
]policy
: {FIFO | RR}When setting this parameter, you must specify a policy. This is one of
FIFO
(first in, first out) orRR
(round robin). The policy value is followed optionally by the priority (an integer).This parameter is available beginning with MySQL Cluster NDB 6.4.0. It is used to determine the total amount of memory to allocate on this node for shared send buffer memory among all configured transporters.
If this parameter is set, its minimum permitted value is 256KB; the maxmimum is 4294967039. For more detailed information about the behavior and use of
TotalSendBufferMemory
and configuring send buffer memory parameters, see Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”.
After making changes in a management node's configuration, it is necessary to perform a rolling restart of the cluster for the new configuration to take effect.
To add new management servers to a running MySQL Cluster, it
is also necessary to perform a rolling restart of all cluster
nodes after modifying any existing
config.ini
files. For more information
about issues arising when using multiple management nodes, see
Section 16.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
The [ndbd]
and [ndbd
default]
sections are used to configure the behavior
of the cluster's data nodes.
[ndbd]
and [ndbd default]
are always used as the section names whether you are using
ndbd or ndbmtd binaries
for the data node processes.
There are many parameters which control buffer sizes, pool
sizes, timeouts, and so forth. The only mandatory parameter is
either one of ExecuteOnComputer
or
HostName
; this must be defined in the local
[ndbd]
section.
The parameter
NoOfReplicas
should be
defined in the [ndbd default]
section, as it
is common to all Cluster data nodes. It is not strictly
necessary to set
NoOfReplicas
, but it is
good practice to set it explicitly.
Most data node parameters are set in the [ndbd
default]
section. Only those parameters explicitly
stated as being able to set local values are permitted to be
changed in the [ndbd]
section. Where present,
HostName
, NodeId
and
ExecuteOnComputer
must
be defined in the local [ndbd]
section, and
not in any other section of config.ini
. In
other words, settings for these parameters are specific to one
data node.
For those parameters affecting memory usage or buffer sizes, it
is possible to use K
, M
,
or G
as a suffix to indicate units of 1024,
1024×1024, or 1024×1024×1024. (For example,
100K
means 100 × 1024 = 102400.)
Parameter names and values are currently case-sensitive.
Information about configuration parameters specific to MySQL Cluster Disk Data tables can be found later in this section.
All of these parameters also apply to ndbmtd
(the multi-threaded version of ndbd). An
additional data node configuration parameter
MaxNoOfExecutionThreads
applies to ndbmtd only, and has no effect
when used with ndbd. For more information,
see Section 16.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.
Identifying data nodes.
The NodeId
or Id
value
(that is, the data node identifier) can be allocated on the
command line when the node is started or in the configuration
file.
Deprecated 5.1.51-ndb-7.1.9 Restart Type node Permitted Values Type numeric
Default Range 1 .. 48
A unique node ID is used as the node's address for all cluster internal messages. For data nodes, this is an integer in the range 1 to 48 inclusive. Each node in the cluster must have a unique identifier.
NodeId
is the preferred parameter name to use when identifying data nodes. Although the olderId
is still supported for backward compatibility, it is now deprecated, and generates a warning when used.Id
is also subject to removal in a future MySQL Cluster release.Restart Type node Permitted Values Type numeric
Default Range 1 .. 48
A unique node ID is used as the node's address for all cluster internal messages. For data nodes, this is an integer in the range 1 to 48 inclusive. Each node in the cluster must have a unique identifier.
NodeId
is the preferred parameter name to use when identifying data nodes. AlthoughId
continues to be supported for backward compatibility, it is now deprecated, generates a warning when used, and is subject to removal in a future version of MySQL Cluster.Restart Type system Permitted Values Type string
Default Range ..
This refers to the
Id
set for one of the computers defined in a[computer]
section.Restart Type system Permitted Values Type string
Default localhost
Range ..
Specifying this parameter defines the hostname of the computer on which the data node is to reside. To specify a hostname other than
localhost
, either this parameter orExecuteOnComputer
is required.Restart Type node Permitted Values Type numeric
Default Range 1 .. 64K
Each node in the cluster uses a port to connect to other nodes. By default, this port is allocated dynamically in such a way as to ensure that no two nodes on the same host computer receive the same port number, so it should normally not be necessary to specify a value for this parameter.
However, if you need to be able to open specific ports in a firewall to permit communication between data nodes and API nodes (including SQL nodes), you can set this parameter to the number of the desired port in an
[ndbd]
section or (if you need to do this for multiple data nodes) the[ndbd default]
section of theconfig.ini
file, and then open the port having that number for incoming connections from SQL nodes, API nodes, or both.ЗамечаниеConnections from data nodes to management nodes is done using the ndb_mgmd management port (the management server's
PortNumber
; see Section 16.3.2.5, “Defining a MySQL Cluster Management Server”) so outgoing connections to that port from any data nodes should always be permitted.Setting this parameter to
TRUE
or1
bindsIP_ADDR_ANY
so that connections can be made from anywhere (for autogenerated connections). The default isFALSE
(0
).Restart Type initial, system Permitted Values Type numeric
Default Range 0 .. 65536
This parameter can be used to assign a data node to a specific node group. It is read only when the cluster is started for the first time, and cannot be used to reassign a data node to a different node group online. It is generally not desirable to use this parameter in the
[ndbd default]
section of theconfig.ini
file, and care must be taken not to assign nodes to node groups in such a way that an invalid numbers of nodes are assigned to any node groups.The
NodeGroup
parameter is chiefly intended for use in adding a new node group to a running MySQL Cluster without having to perform a rolling restart. For this purpose, you should set it to 65536 (the maximum value). You are not required to set aNodeGroup
value for all cluster data nodes, only for those nodes which are to be started and added to the cluster as a new node group at a later time. For more information, see Section 16.5.12.3, “Adding MySQL Cluster Data Nodes Online: Detailed Пример”.Restart Type initial, system Permitted Values Type numeric
Default None
Range 1 .. 4
Permitted Values Type numeric
Default None
Range 1 .. 4
Permitted Values Type numeric
Default 2
Range 1 .. 4
Permitted Values Type numeric
Default 2
Range 1 .. 4
This global parameter can be set only in the
[ndbd default]
section, and defines the number of replicas for each table stored in the cluster. This parameter also specifies the size of node groups. A node group is a set of nodes all storing the same information.Node groups are formed implicitly. The first node group is formed by the set of data nodes with the lowest node IDs, the next node group by the set of the next lowest node identities, and so on. By way of example, assume that we have 4 data nodes and that
NoOfReplicas
is set to 2. The four data nodes have node IDs 2, 3, 4 and 5. Then the first node group is formed from nodes 2 and 3, and the second node group by nodes 4 and 5. It is important to configure the cluster in such a manner that nodes in the same node groups are not placed on the same computer because a single hardware failure would cause the entire cluster to fail.If no node IDs are provided, the order of the data nodes will be the determining factor for the node group. Whether or not explicit assignments are made, they can be viewed in the output of the management client's
SHOW
command.The default value for
NoOfReplicas
is 2, which is the recommended setting in most common usage scenarios.The maximum possible value is 4; currently, only the values 1 and 2 are actually supported.
ImportantSetting
NoOfReplicas
to 1 means that there is only a single copy of all Cluster data; in this case, the loss of a single data node causes the cluster to fail because there are no additional copies of the data stored by that node.The value for this parameter must divide evenly into the number of data nodes in the cluster. For example, if there are two data nodes, then
NoOfReplicas
must be equal to either 1 or 2, since 2/3 and 2/4 both yield fractional values; if there are four data nodes, thenNoOfReplicas
must be equal to 1, 2, or 4.Restart Type initial, node Permitted Values Type string
Default .
Range ..
This parameter specifies the directory where trace files, log files, pid files and error logs are placed.
The default is the data node process working directory.
Restart Type initial, node Permitted Values Type string
Default DataDir
Range ..
This parameter specifies the directory where all files created for metadata, REDO logs, UNDO logs (for Disk Data tables), and data files are placed. The default is the directory specified by
DataDir
.ЗамечаниеThis directory must exist before the ndbd process is initiated.
The recommended directory hierarchy for MySQL Cluster includes
/var/lib/mysql-cluster
, under which a directory for the node's file system is created. The name of this subdirectory contains the node ID. For example, if the node ID is 2, this subdirectory is namedndb_2_fs
.Restart Type initial, node Permitted Values Type string
Default FileSystemPath
Range ..
This parameter specifies the directory in which backups are placed.
ImportantThe string '
/BACKUP
' is always appended to this value. For example, if you set the value ofBackupDataDir
to/var/lib/cluster-data
, then all backups are stored under/var/lib/cluster-data/BACKUP
. This also means that the effective default backup location is the directory namedBACKUP
under the location specified by theFileSystemPath
parameter.
Data Memory, Index Memory, and String Memory
DataMemory
and
IndexMemory
are
[ndbd]
parameters specifying the size of
memory segments used to store the actual records and their
indexes. In setting values for these, it is important to
understand how
DataMemory
and
IndexMemory
are used, as
they usually need to be updated to reflect actual usage by the
cluster:
Restart Type node Permitted Values Type numeric
Default 80M
Range 1M .. 1024G
This parameter defines the amount of space (in bytes) available for storing database records. The entire amount specified by this value is allocated in memory, so it is extremely important that the machine has sufficient physical memory to accommodate it.
The memory allocated by
DataMemory
is used to store both the actual records and indexes. There is a 16-byte overhead on each record; an additional amount for each record is incurred because it is stored in a 32KB page with 128 byte page overhead (see below). There is also a small amount wasted per page due to the fact that each record is stored in only one page.For variable-size table attributes, the data is stored on separate datapages, allocated from
DataMemory
. Variable-length records use a fixed-size part with an extra overhead of 4 bytes to reference the variable-size part. The variable-size part has 2 bytes overhead plus 2 bytes per attribute.The maximum record size is 14000 bytes.
The memory space defined by
DataMemory
is also used to store ordered indexes, which use about 10 bytes per record. Each table row is represented in the ordered index. A common error among users is to assume that all indexes are stored in the memory allocated byIndexMemory
, but this is not the case: Only primary key and unique hash indexes use this memory; ordered indexes use the memory allocated byDataMemory
. However, creating a primary key or unique hash index also creates an ordered index on the same keys, unless you specifyUSING HASH
in the index creation statement. This can be verified by running ndb_desc -ddb_name
table_name
in the management client.Currently, MySQL Cluster can use a maximum of 512 MB for hash indexes per partition, which means in some cases it is possible to get Table is full errors in MySQL client applications even when ndb_mgm -e "ALL REPORT MEMORYUSAGE" shows significant free
DataMemory
. This can also pose a problem with data node restarts on nodes that are heavily loaded with data. You can forceNDB
to create extra partitions for MySQL Cluster tables and thus have more memory available for hash indexes by using theMAX_ROWS
option forCREATE TABLE
. In general, settingMAX_ROWS
to twice the number of rows that you expect to store in the table should be sufficient. In MySQL Cluster 7.2.3 and later, you can also use theMinFreePct
configuration parameter to help avoid problems with node restarts. (Bug #13436216)The memory space allocated by
DataMemory
consists of 32KB pages, which are allocated to table fragments. Each table is normally partitioned into the same number of fragments as there are data nodes in the cluster. Thus, for each node, there are the same number of fragments as are set inNoOfReplicas
.Once a page has been allocated, it is currently not possible to return it to the pool of free pages, except by deleting the table. (This also means that
DataMemory
pages, once allocated to a given table, cannot be used by other tables.) Performing a data node recovery also compresses the partition because all records are inserted into empty partitions from other live nodes.The
DataMemory
memory space also contains UNDO information: For each update, a copy of the unaltered record is allocated in theDataMemory
. There is also a reference to each copy in the ordered table indexes. Unique hash indexes are updated only when the unique index columns are updated, in which case a new entry in the index table is inserted and the old entry is deleted upon commit. For this reason, it is also necessary to allocate enough memory to handle the largest transactions performed by applications using the cluster. In any case, performing a few large transactions holds no advantage over using many smaller ones, for the following reasons:Large transactions are not any faster than smaller ones
Large transactions increase the number of operations that are lost and must be repeated in event of transaction failure
Large transactions use more memory
The default value for
DataMemory
is 80MB; the minimum is 1MB. There is no maximum size, but in reality the maximum size has to be adapted so that the process does not start swapping when the limit is reached. This limit is determined by the amount of physical RAM available on the machine and by the amount of memory that the operating system may commit to any one process. 32-bit operating systems are generally limited to 2–4GB per process; 64-bit operating systems can use more. For large databases, it may be preferable to use a 64-bit operating system for this reason.Restart Type node Permitted Values Type numeric
Default 18M
Range 1M .. 1T
This parameter controls the amount of storage used for hash indexes in MySQL Cluster. Hash indexes are always used for primary key indexes, unique indexes, and unique constraints. Note that when defining a primary key and a unique index, two indexes will be created, one of which is a hash index used for all tuple accesses as well as lock handling. It is also used to enforce unique constraints.
The size of the hash index is 25 bytes per record, plus the size of the primary key. For primary keys larger than 32 bytes another 8 bytes is added.
The default value for
IndexMemory
is 18MB. The minimum is 1MB.Restart Type system Permitted Values (>= 5.5) Type numeric
Default 25
Range 0 .. 4G
This parameter determines how much memory is allocated for strings such as table names, and is specified in an
[ndbd]
or[ndbd default]
section of theconfig.ini
file. A value between0
and100
inclusive is interpreted as a percent of the maximum default value, which is calculated based on a number of factors including the number of tables, maximum table name size, maximum size of.FRM
files,MaxNoOfTriggers
, maximum column name size, and maximum default column value.A value greater than
100
is interpreted as a number of bytes.The default value is 25—that is, 25 percent of the default maximum.
Under most circumstances, the default value should be sufficient, but when you have a great many Cluster tables (1000 or more), it is possible to get Error 773 Out of string memory, please modify StringMemory config parameter: Permanent error: Schema error, in which case you should increase this value.
25
(25 percent) is not excessive, and should prevent this error from recurring in all but the most extreme conditions.
The following example illustrates how memory is used for a table. Consider this table definition:
CREATE TABLE example ( a INT NOT NULL, b INT NOT NULL, c INT NOT NULL, PRIMARY KEY(a), UNIQUE(b) ) ENGINE=NDBCLUSTER;
For each record, there are 12 bytes of data plus 12 bytes
overhead. Having no nullable columns saves 4 bytes of overhead.
In addition, we have two ordered indexes on columns
a
and b
consuming roughly
10 bytes each per record. There is a primary key hash index on
the base table using roughly 29 bytes per record. The unique
constraint is implemented by a separate table with
b
as primary key and a
as
a column. This other table consumes an additional 29 bytes of
index memory per record in the example
table
as well 8 bytes of record data plus 12 bytes of overhead.
Thus, for one million records, we need 58MB for index memory to handle the hash indexes for the primary key and the unique constraint. We also need 64MB for the records of the base table and the unique index table, plus the two ordered index tables.
You can see that hash indexes takes up a fair amount of memory space; however, they provide very fast access to the data in return. They are also used in MySQL Cluster to handle uniqueness constraints.
Currently, the only partitioning algorithm is hashing and ordered indexes are local to each node. Thus, ordered indexes cannot be used to handle uniqueness constraints in the general case.
An important point for both
IndexMemory
and
DataMemory
is that the
total database size is the sum of all data memory and all index
memory for each node group. Each node group is used to store
replicated information, so if there are four nodes with two
replicas, there will be two node groups. Thus, the total data
memory available is 2 ×
DataMemory
for each data
node.
It is highly recommended that
DataMemory
and
IndexMemory
be set to
the same values for all nodes. Data distribution is even over
all nodes in the cluster, so the maximum amount of space
available for any node can be no greater than that of the
smallest node in the cluster.
DataMemory
and
IndexMemory
can be
changed, but decreasing either of these can be risky; doing so
can easily lead to a node or even an entire MySQL Cluster that
is unable to restart due to there being insufficient memory
space. Increasing these values should be acceptable, but it is
recommended that such upgrades are performed in the same manner
as a software upgrade, beginning with an update of the
configuration file, and then restarting the management server
followed by restarting each data node in turn.
Beginning with MySQL Cluster NDB 7.2.3, a proportion (5% by
default) of data node resources including
DataMemory
and
IndexMemory
is kept in
reserve to insure that the data node does not exhaust its memory
when performing a restart. This can be adjusted using the
MinFreePct
data node
configuration parameter (default 5) introduced in the same
version of MySQL Cluster.
Version Introduced | 5.5.17-ndb-7.2.3 | ||
Restart Type | node | ||
Permitted Values | |||
Type | numeric | ||
Default | 5 | ||
Range | 0 .. 100 | ||
Permitted Values | |||
Type | numeric | ||
Default | 5 | ||
Range | 0 .. 100 | ||
Permitted Values | |||
Type | numeric | ||
Default | 5 | ||
Range | 0 .. 100 |
Updates do not increase the amount of index memory used. Inserts take effect immediately; however, rows are not actually deleted until the transaction is committed.
Transaction parameters.
The next few [ndbd]
parameters that we
discuss are important because they affect the number of
parallel transactions and the sizes of transactions that can
be handled by the system.
MaxNoOfConcurrentTransactions
sets the number of parallel transactions possible in a node.
MaxNoOfConcurrentOperations
sets the number of records that can be in update phase or
locked simultaneously.
Both of these parameters (especially
MaxNoOfConcurrentOperations
)
are likely targets for users setting specific values and not
using the default value. The default value is set for systems
using small transactions, to ensure that these do not use
excessive memory.
MaxDMLOperationsPerTransaction
sets the maximum number of DML operations that can be performed
in a given transaction.
Restart Type system Permitted Values Type numeric
Default 4096
Range 32 .. 4G
Each cluster data node requires a transaction record for each active transaction in the cluster. The task of coordinating transactions is distributed among all of the data nodes. The total number of transaction records in the cluster is the number of transactions in any given node times the number of nodes in the cluster.
Transaction records are allocated to individual MySQL servers. Each connection to a MySQL server requires at least one transaction record, plus an additional transaction object per table accessed by that connection. This means that a reasonable minimum for this parameter is
MaxNoOfConcurrentTransactions = (maximum number of tables accessed in any single transaction + 1) * number of cluster SQL nodes
Suppose that there are 4 SQL nodes using the cluster. A single join involving 5 tables requires 6 transaction records; if there are 5 such joins in a transaction, then 5 * 6 = 30 transaction records are required for this transaction, per MySQL server, or 30 * 4 = 120 transaction records total.
This parameter must be set to the same value for all cluster data nodes. This is due to the fact that, when a data node fails, the oldest surviving node re-creates the transaction state of all transactions that were ongoing in the failed node.
Changing the value of
MaxNoOfConcurrentTransactions
requires a complete shutdown and restart of the cluster.The default value is 4096.
Restart Type node Permitted Values Type numeric
Default 32K
Range 32 .. 4G
It is a good idea to adjust the value of this parameter according to the size and number of transactions. When performing transactions of only a few operations each and not involving a great many records, there is no need to set this parameter very high. When performing large transactions involving many records need to set this parameter higher.
Records are kept for each transaction updating cluster data, both in the transaction coordinator and in the nodes where the actual updates are performed. These records contain state information needed to find UNDO records for rollback, lock queues, and other purposes.
This parameter should be set to the number of records to be updated simultaneously in transactions, divided by the number of cluster data nodes. For example, in a cluster which has four data nodes and which is expected to handle 1,000,000 concurrent updates using transactions, you should set this value to 1000000 / 4 = 250000.
Read queries which set locks also cause operation records to be created. Some extra space is allocated within individual nodes to accommodate cases where the distribution is not perfect over the nodes.
When queries make use of the unique hash index, there are actually two operation records used per record in the transaction. The first record represents the read in the index table and the second handles the operation on the base table.
The default value is 32768.
This parameter actually handles two values that can be configured separately. The first of these specifies how many operation records are to be placed with the transaction coordinator. The second part specifies how many operation records are to be local to the database.
A very large transaction performed on an eight-node cluster requires as many operation records in the transaction coordinator as there are reads, updates, and deletes involved in the transaction. However, the operation records of the are spread over all eight nodes. Thus, if it is necessary to configure the system for one very large transaction, it is a good idea to configure the two parts separately.
MaxNoOfConcurrentOperations
will always be used to calculate the number of operation records in the transaction coordinator portion of the node.It is also important to have an idea of the memory requirements for operation records. These consume about 1KB per record.
Restart Type node Permitted Values Type numeric
Default UNDEFINED
Range 32 .. 4G
By default, this parameter is calculated as 1.1 ×
MaxNoOfConcurrentOperations
. This fits systems with many simultaneous transactions, none of them being very large. If there is a need to handle one very large transaction at a time and there are many nodes, it is a good idea to override the default value by explicitly specifying this parameter.MaxDMLOperationsPerTransaction
Restart Type node Permitted Values Type numeric
Default 4294967295
Range 32 .. 4294967295
Permitted Values Type numeric
Default 4294967295
Range 32 .. 4294967295
This parameter limits the size of a transaction. The transaction is aborted if it requires more than this many DML operations. The minimum number of operations per transaction is 32; however, you can set
MaxDMLOperationsPerTransaction
to 0 to disable any limitation on the number of DML operations per transaction. The maximum (and default) is 4294967295.
Transaction temporary storage.
The next set of [ndbd]
parameters is used
to determine temporary storage when executing a statement that
is part of a Cluster transaction. All records are released
when the statement is completed and the cluster is waiting for
the commit or rollback.
The default values for these parameters are adequate for most situations. However, users with a need to support transactions involving large numbers of rows or operations may need to increase these values to enable better parallelism in the system, whereas users whose applications require relatively small transactions can decrease the values to save memory.
MaxNoOfConcurrentIndexOperations
Restart Type node Permitted Values Type numeric
Default 8K
Range 0 .. 4G
For queries using a unique hash index, another temporary set of operation records is used during a query's execution phase. This parameter sets the size of that pool of records. Thus, this record is allocated only while executing a part of a query. As soon as this part has been executed, the record is released. The state needed to handle aborts and commits is handled by the normal operation records, where the pool size is set by the parameter
MaxNoOfConcurrentOperations
.The default value of this parameter is 8192. Only in rare cases of extremely high parallelism using unique hash indexes should it be necessary to increase this value. Using a smaller value is possible and can save memory if the DBA is certain that a high degree of parallelism is not required for the cluster.
Restart Type node Permitted Values Type numeric
Default 4000
Range 0 .. 4G
The default value of
MaxNoOfFiredTriggers
is 4000, which is sufficient for most situations. In some cases it can even be decreased if the DBA feels certain the need for parallelism in the cluster is not high.A record is created when an operation is performed that affects a unique hash index. Inserting or deleting a record in a table with unique hash indexes or updating a column that is part of a unique hash index fires an insert or a delete in the index table. The resulting record is used to represent this index table operation while waiting for the original operation that fired it to complete. This operation is short-lived but can still require a large number of records in its pool for situations with many parallel write operations on a base table containing a set of unique hash indexes.
Restart Type node Permitted Values Type numeric
Default 1M
Range 1K .. 4G
The memory affected by this parameter is used for tracking operations fired when updating index tables and reading unique indexes. This memory is used to store the key and column information for these operations. It is only very rarely that the value for this parameter needs to be altered from the default.
The default value for
TransactionBufferMemory
is 1MB.Normal read and write operations use a similar buffer, whose usage is even more short-lived. The compile-time parameter
ZATTRBUF_FILESIZE
(found inndb/src/kernel/blocks/Dbtc/Dbtc.hpp
) set to 4000 × 128 bytes (500KB). A similar buffer for key information,ZDATABUF_FILESIZE
(also inDbtc.hpp
) contains 4000 × 16 = 62.5KB of buffer space.Dbtc
is the module that handles transaction coordination.
Scans and buffering.
There are additional [ndbd]
parameters in
the Dblqh
module (in
ndb/src/kernel/blocks/Dblqh/Dblqh.hpp
)
that affect reads and updates. These include
ZATTRINBUF_FILESIZE
, set by default to
10000 × 128 bytes (1250KB) and
ZDATABUF_FILE_SIZE
, set by default to
10000*16 bytes (roughly 156KB) of buffer space. To date, there
have been neither any reports from users nor any results from
our own extensive tests suggesting that either of these
compile-time limits should be increased.
Restart Type node Permitted Values Type numeric
Default 256
Range 2 .. 500
This parameter is used to control the number of parallel scans that can be performed in the cluster. Each transaction coordinator can handle the number of parallel scans defined for this parameter. Each scan query is performed by scanning all partitions in parallel. Each partition scan uses a scan record in the node where the partition is located, the number of records being the value of this parameter times the number of nodes. The cluster should be able to sustain
MaxNoOfConcurrentScans
scans concurrently from all nodes in the cluster.Scans are actually performed in two cases. The first of these cases occurs when no hash or ordered indexes exists to handle the query, in which case the query is executed by performing a full table scan. The second case is encountered when there is no hash index to support the query but there is an ordered index. Using the ordered index means executing a parallel range scan. The order is kept on the local partitions only, so it is necessary to perform the index scan on all partitions.
The default value of
MaxNoOfConcurrentScans
is 256. The maximum value is 500.Restart Type node Permitted Values Type numeric
Default UNDEFINED
Range 32 .. 4G
Specifies the number of local scan records if many scans are not fully parallelized. In MySQL Cluster NDB 7.2.0 and later, when the number of local scan records is not provided, it is calculated as 4 times the product of
MaxNoOfConcurrentScans
and the number of data nodes in the system. (Previously, it was calculated as the product ofMaxNoOfConcurrentScans
and the number of data nodes.) The minimum value is 32.Restart Type node Permitted Values (>= 5.5) Type numeric
Default 256
Range 1 .. 992
This parameter is used to calculate the number of lock records used to handle concurrent scan operations.
The default value is 64; this value has a strong connection to the
BatchSize
defined in the SQL nodes.Restart Type node Permitted Values Type numeric
Default 1M
Range 512K .. 4G
Permitted Values Type numeric
Default 4M
Range 512K .. 4G
This is an internal buffer used for passing messages within individual nodes and between nodes. The default is 4MB.
This parameter seldom needs to be changed from the default.
Restart Type node Permitted Values Type numeric
Default 32
Range 1 .. 1G
Permitted Values Type numeric
Default 32
Range 1 .. 1G
Permitted Values Type numeric
Default 256
Range 1 .. 1G
It is possible to copnfigure the maximum number of parallel scans (
TUP
scans andTUX
scans) allowed before they begin queuing for serial handling. You can increase this to take advantage of any unused CPU when performing large number of scans in parallel and improve their performance.Beginning with MySQL Cluster NDB 7.2.0, the default value for this parameter was increased from 32 to 256.
Memory Allocation
This is the maximum size of the memory unit to use when
allocating memory for tables. In cases where
NDB
gives Out of
memory errors, but it is evident by examining the
cluster logs or the output of DUMP 1000
(see
DUMP 1000
) that all
available memory has not yet been used, you can increase the
value of this parameter (or
MaxNoOfTables
, or both)
to cause NDB
to make sufficient
memory available.
Logging and checkpointing.
The following [ndbd]
parameters control log
and checkpoint behavior.
Restart Type initial, node Permitted Values Type numeric
Default 16
Range 3 .. 4G
This parameter sets the number of REDO log files for the node, and thus the amount of space allocated to REDO logging. Because the REDO log files are organized in a ring, it is extremely important that the first and last log files in the set (sometimes referred to as the “head” and “tail” log files, respectively) do not meet. When these approach one another too closely, the node begins aborting all transactions encompassing updates due to a lack of room for new log records.
A
REDO
log record is not removed until the required number of local checkpoints has been completed since that log record was inserted. (In MySQL Cluster NDB 7.2, only 2 local checkpoints are necessary). Checkpointing frequency is determined by its own set of configuration parameters discussed elsewhere in this chapter.The default parameter value is 16, which by default means 16 sets of 4 16MB files for a total of 1024MB. The size of the individual log files is configurable using the
FragmentLogFileSize
parameter. In scenarios requiring a great many updates, the value forNoOfFragmentLogFiles
may need to be set as high as 300 or even higher to provide sufficient space for REDO logs.If the checkpointing is slow and there are so many writes to the database that the log files are full and the log tail cannot be cut without jeopardizing recovery, all updating transactions are aborted with internal error code 410 (
Out of log file space temporarily
). This condition prevails until a checkpoint has completed and the log tail can be moved forward.ImportantThis parameter cannot be changed “on the fly”; you must restart the node using
--initial
. If you wish to change this value for all data nodes in a running cluster, you can do so using a rolling node restart (using--initial
when starting each data node).Restart Type initial, node Permitted Values Type numeric
Default 16M
Range 4M .. 1G
Setting this parameter enables you to control directly the size of redo log files. This can be useful in situations when MySQL Cluster is operating under a high load and it is unable to close fragment log files quickly enough before attempting to open new ones (only 2 fragment log files can be open at one time); increasing the size of the fragment log files gives the cluster more time before having to open each new fragment log file. The default value for this parameter is 16M.
For more information about fragment log files, see the description for
NoOfFragmentLogFiles
.Restart Type initial, node Permitted Values Type string
Default Range ..
By default, fragment log files are created sparsely when performing an initial start of a data node—that is, depending on the operating system and file system in use, not all bytes are necessarily written to disk. However, it is possible to override this behavior and force all bytes to be written, regardless of the platform and file system type being used, by means of this parameter.
InitFragmentLogFiles
takes either of two values:SPARSE
. Fragment log files are created sparsely. This is the default value.FULL
. Force all bytes of the fragment log file to be written to disk.
Depending on your operating system and file system, setting
InitFragmentLogFiles=FULL
may help eliminate I/O errors on writes to the REDO log.Restart Type node Permitted Values Type numeric
Default 0
Range 20 .. 4G
This parameter sets a ceiling on how many internal threads to allocate for open files. Any situation requiring a change in this parameter should be reported as a bug.
The default value is 0. However, the minimum value to which this parameter can be set is 20.
Restart Type node Permitted Values Type numeric
Default 27
Range 20 .. 4G
This parameter sets the initial number of internal threads to allocate for open files.
The default value is 27.
Restart Type node Permitted Values Type numeric
Default 25
Range 0 .. 4G
This parameter sets the maximum number of trace files that are kept before overwriting old ones. Trace files are generated when, for whatever reason, the node crashes.
The default is 25 trace files.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 600
Permitted Values Type numeric
Default 0
Range 0 .. 600
In parallel data node recovery, only table data is actually copied and synchronized in parallel; synchronization of metadata such as dictionary and checkpoint information is done in a serial fashion. In addition, recovery of dictionary and checkpoint information cannot be executed in parallel with performing of local checkpoints. This means that, when starting or restarting many data nodes concurrently, data nodes may be forced to wait while a local checkpoint is performed, which can result in longer node recovery times.
It is possible to force a delay in the local checkpoint to permit more (and possibly all) data nodes to complete metadata synchronization; once each data node's metadata synchronization is complete, all of the data nodes can recover table data in parallel, even while the local checkpoint is being executed. To force such a delay, set
MaxLCPStartDelay
, which determines the number of seconds the cluster can wait to begin a local checkpoint while data nodes continue to synchronize metadata. This parameter should be set in the[ndbd default]
section of theconfig.ini
file, so that it is the same for all data nodes. The maximum value is 600; the default is 0.
Metadata objects.
The next set of [ndbd]
parameters defines
pool sizes for metadata objects, used to define the maximum
number of attributes, tables, indexes, and trigger objects
used by indexes, events, and replication between clusters.
Note that these act merely as “suggestions” to
the cluster, and any that are not specified revert to the
default values shown.
Restart Type node Permitted Values Type numeric
Default 1000
Range 32 .. 4G
This parameter sets a suggested maximum number of attributes that can be defined in the cluster; like
MaxNoOfTables
, it is not intended to function as a hard upper limit.(In older MySQL Cluster releases, this parameter was sometimes treated as a hard limit for certain operations. This caused problems with MySQL Cluster Replication, when it was possible to create more tables than could be replicated, and sometimes led to confusion when it was possible [or not possible, depending on the circumstances] to create more than
MaxNoOfAttributes
attributes.)The default value is 1000, with the minimum possible value being 32. The maximum is 4294967039. Each attribute consumes around 200 bytes of storage per node due to the fact that all metadata is fully replicated on the servers.
When setting
MaxNoOfAttributes
, it is important to prepare in advance for anyALTER TABLE
statements that you might want to perform in the future. This is due to the fact, during the execution ofALTER TABLE
on a Cluster table, 3 times the number of attributes as in the original table are used, and a good practice is to permit double this amount. For example, if the MySQL Cluster table having the greatest number of attributes (greatest_number_of_attributes
) has 100 attributes, a good starting point for the value ofMaxNoOfAttributes
would be6 *
.greatest_number_of_attributes
= 600You should also estimate the average number of attributes per table and multiply this by
MaxNoOfTables
. If this value is larger than the value obtained in the previous paragraph, you should use the larger value instead.Assuming that you can create all desired tables without any problems, you should also verify that this number is sufficient by trying an actual
ALTER TABLE
after configuring the parameter. If this is not successful, increaseMaxNoOfAttributes
by another multiple ofMaxNoOfTables
and test it again.Restart Type node Permitted Values Type numeric
Default 128
Range 8 .. 20320
A table object is allocated for each table and for each unique hash index in the cluster. This parameter sets a suggested maximum number of table objects for the cluster as a whole; like
MaxNoOfAttributes
, it is not intended to function as a hard upper limit.(In older MySQL Cluster releases, this parameter was sometimes treated as a hard limit for certain operations. This caused problems with MySQL Cluster Replication, when it was possible to create more tables than could be replicated, and sometimes led to confusion when it was possible [or not possible, depending on the circumstances] to create more than
MaxNoOfTables
tables.)For each attribute that has a
BLOB
data type an extra table is used to store most of theBLOB
data. These tables also must be taken into account when defining the total number of tables.The default value of this parameter is 128. The minimum is 8 and the maximum is 20320. Each table object consumes approximately 20KB per node.
ЗамечаниеThe sum of
MaxNoOfTables
,MaxNoOfOrderedIndexes
, andMaxNoOfUniqueHashIndexes
must not exceed232 – 2
(4294967294).Restart Type node Permitted Values Type numeric
Default 128
Range 0 .. 4G
For each ordered index in the cluster, an object is allocated describing what is being indexed and its storage segments. By default, each index so defined also defines an ordered index. Each unique index and primary key has both an ordered index and a hash index.
MaxNoOfOrderedIndexes
sets the total number of ordered indexes that can be in use in the system at any one time.The default value of this parameter is 128. Each index object consumes approximately 10KB of data per node.
ЗамечаниеThe sum of
MaxNoOfTables
,MaxNoOfOrderedIndexes
, andMaxNoOfUniqueHashIndexes
must not exceed232 – 2
(4294967294).Restart Type node Permitted Values Type numeric
Default 64
Range 0 .. 4G
For each unique index that is not a primary key, a special table is allocated that maps the unique key to the primary key of the indexed table. By default, an ordered index is also defined for each unique index. To prevent this, you must specify the
USING HASH
option when defining the unique index.The default value is 64. Each index consumes approximately 15KB per node.
ЗамечаниеThe sum of
MaxNoOfTables
,MaxNoOfOrderedIndexes
, andMaxNoOfUniqueHashIndexes
must not exceed232 – 2
(4294967294).Restart Type node Permitted Values Type numeric
Default 768
Range 0 .. 4G
Internal update, insert, and delete triggers are allocated for each unique hash index. (This means that three triggers are created for each unique hash index.) However, an ordered index requires only a single trigger object. Backups also use three trigger objects for each normal table in the cluster.
Replication between clusters also makes use of internal triggers.
This parameter sets the maximum number of trigger objects in the cluster.
The default value is 768.
This parameter is deprecated in MySQL Cluster 5.5 and later; you should use
MaxNoOfOrderedIndexes
andMaxNoOfUniqueHashIndexes
instead.This parameter is used only by unique hash indexes. There needs to be one record in this pool for each unique hash index defined in the cluster.
The default value of this parameter is 128.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
Permitted Values Type numeric
Default 0
Range 0 .. 4G
Each
NDB
table in a MySQL Cluster requires a subscription in the NDB kernel. For some NDB API applications, it may be necessary or desirable to change this paramete. However, for normal usage with MySQL servers acting as SQL nodes, there is not any need to do so.The default value for
MaxNoOfSubscriptions
is 0, which is treated as equal toMaxNoOfTables
. Each subscription consumes 108 bytes.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
Permitted Values Type numeric
Default 0
Range 0 .. 4G
This parameter is of interest only when using MySQL Cluster Replication. The default value is 0, which is treated as
2 * MaxNoOfTables
; that is, there is one subscription perNDB
table for each of two MySQL servers (one acting as the replication master and the other as the slave). Each subscriber uses 16 bytes of memory.When using circular replication, multi-master replcation, and other replication setups involving more than 2 MySQL servers, you should increase this parameter to the number of mysqld processes included in replication (this is often, but not always, the same as the number of clusters). For example, if you have a circular replication setup using three MySQL Clusters, with one mysqld attached to each cluster, and each of these mysqld processes acts as a master and as a slave, you should set
MaxNoOfSubscribers
equal to3 * MaxNoOfTables
.For more information, see Section 16.6, “MySQL Cluster Replication”.
MaxNoOfConcurrentSubOperations
Restart Type node Permitted Values Type numeric
Default 256
Range 0 .. 4G
Permitted Values Type numeric
Default 256
Range 0 .. 4G
This parameter sets a ceiling on the number of operations that can be performed by all API nodes in the cluster at one time. The default value (256) is sufficient for normal operations, and might need to be adjusted only in scenarios where there are a great many API nodes each performing a high volume of operations concurrently.
Boolean parameters.
The behavior of data nodes is also affected by a set of
[ndbd]
parameters taking on boolean values.
These parameters can each be specified as
TRUE
by setting them equal to
1
or Y
, and as
FALSE
by setting them equal to
0
or N
.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 2
For a number of operating systems, including Solaris and Linux, it is possible to lock a process into memory and so avoid any swapping to disk. This can be used to help guarantee the cluster's real-time characteristics.
This parameter takes one of the integer values
0
,1
, or2
, which act as shown in the following list:0
: Disables locking. This is the default value.1
: Performs the lock after allocating memory for the process.2
: Performs the lock before memory for the process is allocated.
If the operating system is not configured to permit unprivileged users to lock pages, then the data node process making use of this parameter may have to be run as system root. (
LockPagesInMainMemory
uses themlockall
function. From Linux kernel 2.6.9, unprivileged users can lock memory as limited bymax locked memory
. For more information, see ulimit -l and http://linux.die.net/man/2/mlock).ЗамечаниеIn older MySQL Cluster releases, this parameter was a Boolean.
0
orfalse
was the default setting, and disabled locking.1
ortrue
enabled locking of the process after its memory was allocated. In MySQL Cluster NDB 7.2, usingtrue
orfalse
as the value of this parameter causes an error.Restart Type node Permitted Values Type boolean
Default true
Range ..
This parameter specifies whether an ndbd process should exit or perform an automatic restart when an error condition is encountered.
This feature is enabled by default.
Version Introduced 5.5.16-ndb-7.2.1 Restart Type Permitted Values Type boolean
Default OFF
Permitted Values Type boolean
Default ON
When this parameter is enabled, it forces a data node to shut down whenever it encounters a corrupted tuple. In MySQL Cluster NDB 7.2.1 and later, it is enabled by default. This is a change from MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1, where it was disabled by default.
Restart Type initial, system Permitted Values Type boolean
Default 0
Range 0 .. 1
It is possible to specify MySQL Cluster tables as diskless, meaning that tables are not checkpointed to disk and that no logging occurs. Such tables exist only in main memory. A consequence of using diskless tables is that neither the tables nor the records in those tables survive a crash. However, when operating in diskless mode, it is possible to run ndbd on a diskless computer.
ImportantThis feature causes the entire cluster to operate in diskless mode.
When this feature is enabled, Cluster online backup is disabled. In addition, a partial start of the cluster is not possible.
Diskless
is disabled by default.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 1
Permitted Values Type numeric
Default 0
Range 0 .. 1
Permitted Values Type numeric
Default 0
Range 0 .. 1
Enabling this parameter causes
NDB
to attempt usingO_DIRECT
writes for LCP, backups, and redo logs, often lowering kswapd and CPU usage. When using MySQL Cluster on Linux, enableODirect
if you are using a 2.6 or later kernel.ODirect
is disabled by default.Restart Type node Permitted Values Type numeric
Default 2
Range 0 .. 4
This feature is accessible only when building the debug version where it is possible to insert errors in the execution of individual blocks of code as part of testing.
This feature is disabled by default.
Restart Type node Permitted Values Type boolean
Default false
Range ..
Setting this parameter to
1
causes backup files to be compressed. The compression used is equivalent to gzip --fast, and can save 50% or more of the space required on the data node to store uncompressed backup files. Compressed backups can be enabled for individual data nodes, or for all data nodes (by setting this parameter in the[ndbd default]
section of theconfig.ini
file).ImportantYou cannot restore a compressed backup to a cluster running a MySQL version that does not support this feature.
The default value is
0
(disabled).Restart Type node Permitted Values Type boolean
Default false
Range ..
Setting this parameter to
1
causes local checkpoint files to be compressed. The compression used is equivalent to gzip --fast, and can save 50% or more of the space required on the data node to store uncompressed checkpoint files. Compressed LCPs can be enabled for individual data nodes, or for all data nodes (by setting this parameter in the[ndbd default]
section of theconfig.ini
file).ImportantYou cannot restore a compressed local checkpoint to a cluster running a MySQL version that does not support this feature.
The default value is
0
(disabled).
Controlling Timeouts, Intervals, and Disk Paging
There are a number of [ndbd]
parameters
specifying timeouts and intervals between various actions in
Cluster data nodes. Most of the timeout values are specified in
milliseconds. Any exceptions to this are mentioned where
applicable.
Restart Type node Permitted Values Type numeric
Default 6000
Range 70 .. 4G
To prevent the main thread from getting stuck in an endless loop at some point, a “watchdog” thread checks the main thread. This parameter specifies the number of milliseconds between checks. If the process remains in the same state after three checks, the watchdog thread terminates it.
This parameter can easily be changed for purposes of experimentation or to adapt to local conditions. It can be specified on a per-node basis although there seems to be little reason for doing so.
The default timeout is 6000 milliseconds (6 seconds).
TimeBetweenWatchDogCheckInitial
Restart Type node Permitted Values Type numeric
Default 6000
Range 70 .. 4G
This is similar to the
TimeBetweenWatchDogCheck
parameter, except thatTimeBetweenWatchDogCheckInitial
controls the amount of time that passes between execution checks inside a database node in the early start phases during which memory is allocated.The default timeout is 6000 milliseconds (6 seconds).
Restart Type node Permitted Values Type numeric
Default 30000
Range 0 .. 4G
This parameter specifies how long the Cluster waits for all data nodes to come up before the cluster initialization routine is invoked. This timeout is used to avoid a partial Cluster startup whenever possible.
This parameter is overridden when performing an initial start or initial restart of the cluster.
The default value is 30000 milliseconds (30 seconds). 0 disables the timeout, in which case the cluster may start only if all nodes are available.
Restart Type node Permitted Values Type numeric
Default 60000
Range 0 .. 4G
If the cluster is ready to start after waiting for
StartPartialTimeout
milliseconds but is still possibly in a partitioned state, the cluster waits until this timeout has also passed. IfStartPartitionedTimeout
is set to 0, the cluster waits indefinitely.This parameter is overridden when performing an initial start or initial restart of the cluster.
The default timeout is 60000 milliseconds (60 seconds).
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
If a data node has not completed its startup sequence within the time specified by this parameter, the node startup fails. Setting this parameter to 0 (the default value) means that no data node timeout is applied.
For nonzero values, this parameter is measured in milliseconds. For data nodes containing extremely large amounts of data, this parameter should be increased. For example, in the case of a data node containing several gigabytes of data, a period as long as 10–15 minutes (that is, 600000 to 1000000 milliseconds) might be required to perform a node restart.
Restart Type node Permitted Values Type numeric
Default 15000
Range 0 .. 4294967039
When a data node is configured with
Nodegroup = 65536
, is regarded as not being assigned to any node group. When that is done, the cluster waitsStartNoNodegroupTimeout
milliseconds, then treats such nodes as though they had been added to the list passed to the--nowait-nodes
option, and starts. The default value is15000
(that is, the management server waits 15 seconds). Setting this parameter equal to0
means that the cluster waits indefinitely.StartNoNodegroupTimeout
must be the same for all data nodes in the cluster; for this reason, you should always set it in the[ndbd default]
section of theconfig.ini
file, rather than for individual data nodes.See Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”, for more information.
Restart Type node Permitted Values Type numeric
Default 1500
Range 10 .. 4G
One of the primary methods of discovering failed nodes is by the use of heartbeats. This parameter states how often heartbeat signals are sent and how often to expect to receive them. After missing three heartbeat intervals in a row, the node is declared dead. Thus, the maximum time for discovering a failure through the heartbeat mechanism is four times the heartbeat interval.
In MySQL Cluster NDB 7.2.0 and later, the default heartbeat interval is 5000 milliseconds (5 seconds). (Previously, the default was 1500 milliseconds [1.5 seconds]). This parameter must not be changed drastically and should not vary widely between nodes. If one node uses 5000 milliseconds and the node watching it uses 1000 milliseconds, obviously the node will be declared dead very quickly. This parameter can be changed during an online software upgrade, but only in small increments.
See also Network communication and latency.
Restart Type node Permitted Values Type numeric
Default 1500
Range 100 .. 4G
Each data node sends heartbeat signals to each MySQL server (SQL node) to ensure that it remains in contact. If a MySQL server fails to send a heartbeat in time it is declared “dead,” in which case all ongoing transactions are completed and all resources released. The SQL node cannot reconnect until all activities initiated by the previous MySQL instance have been completed. The three-heartbeat criteria for this determination are the same as described for
HeartbeatIntervalDbDb
.The default interval is 1500 milliseconds (1.5 seconds). This interval can vary between individual data nodes because each data node watches the MySQL servers connected to it, independently of all other data nodes.
For more information, see Network communication and latency.
Restart Type system Permitted Values Type numeric
Default 0
Range 0 .. 65535
Data nodes send heartbeats to one another in a circular fashion whereby each data node monitors the previous one. If a heartbeat is not detected by a given data node, this node declares the previous data node in the circle “dead” (that is, no longer accessible by the cluster). The determination that a data node is dead is done globally; in other words; once a data node is declared dead, it is regarded as such by all nodes in the cluster.
It is possible for heartbeats between data nodes residing on different hosts to be too slow compared to heartbeats between other pairs of nodes (for example, due to a very low heartbeat interval or temporary connection problem), such that a data node is declared dead, even though the node can still function as part of the cluster. .
In this type of situation, it may be that the order in which heartbeats are transmitted between data nodes makes a difference as to whether or not a particular data node is declared dead. If this declaration occurs unnecessarily, this can in turn lead to the unnecessary loss of a node group and as thus to a failure of the cluster.
Consider a setup where there are 4 data nodes A, B, C, and D running on 2 host computers
host1
andhost2
, and that these data nodes make up 2 node groups, as shown in the following table:host1
host2
Node Group 0: Node A Node B Node Group 1: Node C Node D Suppose the heartbeats are transmitted in the order A->B->C->D->A. In this case, the loss of the heartbeat between the hosts causes node B to declare node A dead and node C to declare node B dead. This results in loss of Node Group 0, and so the cluster fails. On the other hand, if the order of transmission is A->B->D->C->A (and all other conditions remain as previously stated), the loss of the heartbeat causes nodes A and D to be declared dead; in this case, each node group has one surviving node, and the cluster survives.
The
HeartbeatOrder
configuration parameter makes the order of heartbeat transmission user-configurable. The default value forHeartbeatOrder
is zero; allowing the default value to be used on all data nodes causes the order of heartbeat transmission to be determined byNDB
. If this parameter is used, it must be set to a nonzero value (maximum 65535) for every data node in the cluster, and this value must be unique for each data node; this causes the heartbeat transmission to proceed from data node to data node in the order of theirHeartbeatOrder
values from lowest to highest (and then directly from the data node having the highestHeartbeatOrder
to the data node having the lowest value, to complete the circle). The values need not be consecutive; for example, to force the heartbeat transmission order A->B->D->C->A in the scenario outlined previously, you could set theHeartbeatOrder
values as shown here:Node HeartbeatOrder
A 10 B 20 C 30 D 25 To use this parameter to change the heartbeat transmission order in a running MySQL Cluster, you must first set
HeartbeatOrder
for each data node in the cluster in the global configuration (config.ini
) file (or files). To cause the change to take effect, you must perform either of the following:A complete shutdown and restart of the entire cluster.
2 rolling restarts of the cluster in succession. All nodes must be restarted in the same order in both rolling restarts.
You can use
DUMP 908
to observe the effect of this parameter in the data node logs.Restart Type node Permitted Values Type string
Default 1500
Range 0 .. 4G
This parameter enables connection checking between data nodes. A data node that fails to respond within an interval of
ConnectCheckIntervalDelay
seconds is considered suspect, and is considered dead after two such intervals.The default value for this parameter is 0; this is a change from MySQL Cluster NDB 7.1.
Restart Type node Permitted Values Type numeric
Default 20
Range 0 .. 31
This parameter is an exception in that it does not specify a time to wait before starting a new local checkpoint; rather, it is used to ensure that local checkpoints are not performed in a cluster where relatively few updates are taking place. In most clusters with high update rates, it is likely that a new local checkpoint is started immediately after the previous one has been completed.
The size of all write operations executed since the start of the previous local checkpoints is added. This parameter is also exceptional in that it is specified as the base-2 logarithm of the number of 4-byte words, so that the default value 20 means 4MB (4 × 220) of write operations, 21 would mean 8MB, and so on up to a maximum value of 31, which equates to 8GB of write operations.
All the write operations in the cluster are added together. Setting
TimeBetweenLocalCheckpoints
to 6 or less means that local checkpoints will be executed continuously without pause, independent of the cluster's workload.Restart Type node Permitted Values Type numeric
Default 2000
Range 10 .. 32000
When a transaction is committed, it is committed in main memory in all nodes on which the data is mirrored. However, transaction log records are not flushed to disk as part of the commit. The reasoning behind this behavior is that having the transaction safely committed on at least two autonomous host machines should meet reasonable standards for durability.
It is also important to ensure that even the worst of cases—a complete crash of the cluster—is handled properly. To guarantee that this happens, all transactions taking place within a given interval are put into a global checkpoint, which can be thought of as a set of committed transactions that has been flushed to disk. In other words, as part of the commit process, a transaction is placed in a global checkpoint group. Later, this group's log records are flushed to disk, and then the entire group of transactions is safely committed to disk on all computers in the cluster.
This parameter defines the interval between global checkpoints. The default is 2000 milliseconds.
Restart Type node Permitted Values Type numeric
Default 100
Range 0 .. 32000
This parameter defines the interval between synchronisation epochs for MySQL Cluster Replication. The default value is 100 milliseconds.
TimeBetweenEpochs
is part of the implementation of “micro-GCPs”, which can be used to improve the performance of MySQL Cluster Replication.Restart Type node Permitted Values Type numeric
Default 4000
Range 0 .. 256000
This parameter defines a timeout for synchronization epochs for MySQL Cluster Replication. If a node fails to participate in a global checkpoint within the time determined by this parameter, the node is shut down. In MySQL Cluster NDB 7.2.0 and later, the default value is 0; in other words, the timeout is disabled. This represents a change from previous versions of MySQL Cluster, in which the default value was 4000 milliseconds (4 seconds).
TimeBetweenEpochsTimeout
is part of the implementation of “micro-GCPs”, which can be used to improve the performance of MySQL Cluster Replication.The current value of this parameter and a warning are written to the cluster log whenever a GCP save takes longer than 1 minute or a GCP save takes longer than 10 seconds.
Setting this parameter to zero has the effect of disabling GCP stops caused by save timeouts, commit timeouts, or both. The maximum possible value for this parameter is 256000 milliseconds.
Restart Type node Permitted Values Type numeric
Default 100
Range 0 .. 100000
The number of unprocessed epochs by which a subscribing node can lag behind. Exceeding this number causes a lagging subscriber to be disconnected.
The default value of 100 is sufficient for most normal operations. If a subscribing node does lag enough to cause disconnections, it is usually due to network or scheduling issues with regard to processes or threads. (In rare circumstances, the problem may be due to a bug in the
NDB
client.) It may be desirable to set the value lower than the default when epochs are longer.Disconnection prevents client issues from affecting the data node service, running out of memory to buffer data, and eventually shutting down. Instead, only the client is affected as a result of the disconnect (by, for example gap events in the binary log), forcing the client to reconnect or restart the process.
TimeBetweenInactiveTransactionAbortCheck
Restart Type node Permitted Values Type numeric
Default 1000
Range 1000 .. 4G
Timeout handling is performed by checking a timer on each transaction once for every interval specified by this parameter. Thus, if this parameter is set to 1000 milliseconds, every transaction will be checked for timing out once per second.
The default value is 1000 milliseconds (1 second).
Restart Type node Permitted Values Type numeric
Default 4G
Range 0 .. 4G
This parameter states the maximum time that is permitted to lapse between operations in the same transaction before the transaction is aborted.
The default for this parameter is
4G
(also the maximum). For a real-time database that needs to ensure that no transaction keeps locks for too long, this parameter should be set to a relatively small value. The unit is milliseconds.TransactionDeadlockDetectionTimeout
Restart Type node Permitted Values Type numeric
Default 1200
Range 50 .. 4G
When a node executes a query involving a transaction, the node waits for the other nodes in the cluster to respond before continuing. A failure to respond can occur for any of the following reasons:
The node is “dead”
The operation has entered a lock queue
The node requested to perform the action could be heavily overloaded.
This timeout parameter states how long the transaction coordinator waits for query execution by another node before aborting the transaction, and is important for both node failure handling and deadlock detection.
The default timeout value is 1200 milliseconds (1.2 seconds).
The minimum for this parameter is 50 milliseconds.
Restart Type node Permitted Values Type numeric
Default 4M
Range 32K .. 4G
This is the maximum number of bytes to store before flushing data to a local checkpoint file. This is done to prevent write buffering, which can impede performance significantly. This parameter is not intended to take the place of
TimeBetweenLocalCheckpoints
.ЗамечаниеWhen
ODirect
is enabled, it is not necessary to setDiskSyncSize
; in fact, in such cases its value is simply ignored.The default value is 4M (4 megabytes).
Restart Type node Permitted Values Type numeric
Default 10M
Range 1M .. 4G
The amount of data,in bytes per second, that is sent to disk during a local checkpoint. This allocation is shared by DML operations and backups (but not backup logging), which means that backups started during times of intensive DML may be impaired by flooding of the redo log buffer and may fail altogether if the contention is sufficiently severe.
The default value is 10M (10 megabytes per second).
Restart Type node Permitted Values Type numeric
Default 100M
Range 1M .. 4G
The amount of data,in bytes per second, that is sent to disk during a local checkpoint as part of a restart operation.
The default value is 100M (100 megabytes per second).
NoOfDiskPagesToDiskAfterRestartTUP
Restart Type node Permitted Values Type numeric
Default 40
Range 1 .. 4G
When executing a local checkpoint, the algorithm flushes all data pages to disk. Merely doing so as quickly as possible without any moderation is likely to impose excessive loads on processors, networks, and disks. To control the write speed, this parameter specifies how many pages per 100 milliseconds are to be written. In this context, a “page” is defined as 8KB. This parameter is specified in units of 80KB per second, so setting
NoOfDiskPagesToDiskAfterRestartTUP
to a value of20
entails writing 1.6MB in data pages to disk each second during a local checkpoint. This value includes the writing of UNDO log records for data pages. That is, this parameter handles the limitation of writes from data memory. (See the entry forIndexMemory
for information about index pages.)In short, this parameter specifies how quickly to execute local checkpoints. It operates in conjunction with
NoOfFragmentLogFiles
,DataMemory
, andIndexMemory
.For more information about the interaction between these parameters and possible strategies for choosing appropriate values for them, see Configuring MySQL Cluster Parameters for Local Checkpoints.
The default value is 40 (3.2MB of data pages per second).
ЗамечаниеThis parameter is deprecated. Use
DiskCheckpointSpeedInRestart
andDiskSyncSize
instead.NoOfDiskPagesToDiskAfterRestartACC
Restart Type node Permitted Values Type numeric
Default 20
Range 1 .. 4G
This parameter uses the same units as
NoOfDiskPagesToDiskAfterRestartTUP
and acts in a similar fashion, but limits the speed of writing index pages from index memory.The default value of this parameter is 20 (1.6MB of index memory pages per second).
ЗамечаниеThis parameter is deprecated. Use
DiskCheckpointSpeedInRestart
andDiskSyncSize
instead.NoOfDiskPagesToDiskDuringRestartTUP
(DEPRECATED)Restart Type node Permitted Values Type numeric
Default 40
Range 1 .. 4G
This parameter is used in a fashion similar to
NoOfDiskPagesToDiskAfterRestartTUP
andNoOfDiskPagesToDiskAfterRestartACC
, only it does so with regard to local checkpoints executed in the node when a node is restarting. A local checkpoint is always performed as part of all node restarts. During a node restart it is possible to write to disk at a higher speed than at other times, because fewer activities are being performed in the node.This parameter covers pages written from data memory.
The default value is 40 (3.2MB per second).
ЗамечаниеThis parameter is deprecated. Use
DiskCheckpointSpeedInRestart
andDiskSyncSize
instead.NoOfDiskPagesToDiskDuringRestartACC
(DEPRECATED)Restart Type node Permitted Values Type numeric
Default 20
Range 1 .. 4G
Controls the number of index memory pages that can be written to disk during the local checkpoint phase of a node restart.
As with
NoOfDiskPagesToDiskAfterRestartTUP
andNoOfDiskPagesToDiskAfterRestartACC
, values for this parameter are expressed in terms of 8KB pages written per 100 milliseconds (80KB/second).The default value is 20 (1.6MB per second).
ЗамечаниеThis parameter is deprecated. Use
DiskCheckpointSpeedInRestart
andDiskSyncSize
instead.Restart Type node Permitted Values Type numeric
Default 3000
Range 10 .. 4G
This parameter specifies how long data nodes wait for a response from the arbitrator to an arbitration message. If this is exceeded, the network is assumed to have split.
In MySQL Cluster NDB 7.2.0 and later, the default value is 7500 milliseconds (7.5 seconds). Previously, this was 3000 milliseconds (3 seconds).
Restart Type node Permitted Values Type enumeration
Default Default
Valid Values Default
Disabled
WaitExternal
The
Arbitration
parameter enables a choice of arbitration schemes, corresponding to one of 3 possible values for this parameter:Default
. This enables arbitration to proceed normally, as determined by theArbitrationRank
settings for the management and API nodes. This is the default value.Disabled
. SettingArbitration = Disabled
in the[ndbd default]
section of theconfig.ini
file to accomplishes the same task as settingArbitrationRank
to 0 on all management and API nodes. WhenArbitration
is set in this way, anyArbitrationRank
settings are ignored.WaitExternal
. TheArbitration
parameter also makes it possible to configure arbitration in such a way that the cluster waits until after the time determined byArbitrationTimeout
has passed for an external cluster manager application to perform arbitration instead of handling arbitration internally. This can be done by settingArbitration = WaitExternal
in the[ndbd default]
section of theconfig.ini
file. For best results with theWaitExternal
setting, it is recommended thatArbitrationTimeout
be 2 times as long as the interval required by the external cluster manager to perform arbitration.
ImportantThis parameter should be used only in the
[ndbd default]
section of the cluster configuration file. The behavior of the cluster is unspecified whenArbitration
is set to different values for individual data nodes.
Buffering and logging.
Several [ndbd]
configuration parameters
enable the advanced user to have more control over the
resources used by node processes and to adjust various buffer
sizes at need.
These buffers are used as front ends to the file system when
writing log records to disk. If the node is running in diskless
mode, these parameters can be set to their minimum values
without penalty due to the fact that disk writes are
“faked” by the NDB
storage engine's file system abstraction layer.
Restart Type node Permitted Values Type numeric
Default 2M
Range 1M .. 4G
The UNDO index buffer, whose size is set by this parameter, is used during local checkpoints. The
NDB
storage engine uses a recovery scheme based on checkpoint consistency in conjunction with an operational REDO log. To produce a consistent checkpoint without blocking the entire system for writes, UNDO logging is done while performing the local checkpoint. UNDO logging is activated on a single table fragment at a time. This optimization is possible because tables are stored entirely in main memory.The UNDO index buffer is used for the updates on the primary key hash index. Inserts and deletes rearrange the hash index; the NDB storage engine writes UNDO log records that map all physical changes to an index page so that they can be undone at system restart. It also logs all active insert operations for each fragment at the start of a local checkpoint.
Reads and updates set lock bits and update a header in the hash index entry. These changes are handled by the page-writing algorithm to ensure that these operations need no UNDO logging.
This buffer is 2MB by default. The minimum value is 1MB, which is sufficient for most applications. For applications doing extremely large or numerous inserts and deletes together with large transactions and large primary keys, it may be necessary to increase the size of this buffer. If this buffer is too small, the NDB storage engine issues internal error code 677 (
Index UNDO buffers overloaded
).ImportantIt is not safe to decrease the value of this parameter during a rolling restart.
Restart Type node Permitted Values Type numeric
Default 16M
Range 1M .. 4G
This parameter sets the size of the UNDO data buffer, which performs a function similar to that of the UNDO index buffer, except the UNDO data buffer is used with regard to data memory rather than index memory. This buffer is used during the local checkpoint phase of a fragment for inserts, deletes, and updates.
Because UNDO log entries tend to grow larger as more operations are logged, this buffer is also larger than its index memory counterpart, with a default value of 16MB.
This amount of memory may be unnecessarily large for some applications. In such cases, it is possible to decrease this size to a minimum of 1MB.
It is rarely necessary to increase the size of this buffer. If there is such a need, it is a good idea to check whether the disks can actually handle the load caused by database update activity. A lack of sufficient disk space cannot be overcome by increasing the size of this buffer.
If this buffer is too small and gets congested, the NDB storage engine issues internal error code 891 (Data UNDO buffers overloaded).
ImportantIt is not safe to decrease the value of this parameter during a rolling restart.
Restart Type node Permitted Values Type numeric
Default 8M
Range 1M .. 4G
All update activities also need to be logged. The REDO log makes it possible to replay these updates whenever the system is restarted. The NDB recovery algorithm uses a “fuzzy” checkpoint of the data together with the UNDO log, and then applies the REDO log to play back all changes up to the restoration point.
RedoBuffer
sets the size of the buffer in which the REDO log is written. The default value is 32MB; the minimum value is 1MB.If this buffer is too small, the
NDB
storage engine issues error code 1221 (REDO log buffers overloaded). For this reason, you should exercise care if you attempt to decrease the value ofRedoBuffer
as part of an online change in the cluster's configuration.
Controlling log messages.
In managing the cluster, it is very important to be able to
control the number of log messages sent for various event
types to stdout
. For each event category,
there are 16 possible event levels (numbered 0 through 15).
Setting event reporting for a given event category to level 15
means all event reports in that category are sent to
stdout
; setting it to 0 means that there
will be no event reports made in that category.
By default, only the startup message is sent to
stdout
, with the remaining event reporting
level defaults being set to 0. The reason for this is that these
messages are also sent to the management server's cluster log.
An analogous set of levels can be set for the management client to determine which event levels to record in the cluster log.
Restart Type node Permitted Values Type numeric
Default 1
Range 0 .. 15
The reporting level for events generated during startup of the process.
The default level is 1.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated as part of graceful shutdown of a node.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for statistical events such as number of primary key reads, number of updates, number of inserts, information relating to buffer usage, and so on.
The default level is 0.
Restart Type initial, node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated by local and global checkpoints.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated during node restart.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated by connections between cluster nodes.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated by errors and warnings by the cluster as a whole. These errors do not cause any node failure but are still considered worth reporting.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated by congestion. These errors do not cause node failure but are still considered worth reporting.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 15
The reporting level for events generated for information about the general state of the cluster.
The default level is 0.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
This parameter controls how often data node memory usage reports are recorded in the cluster log; it is an integer value representing the number of seconds between reports.
Each data node's data memory and index memory usage is logged as both a percentage and a number of 32 KB pages of the
DataMemory
andIndexMemory
, respectively, set in theconfig.ini
file. For example, ifDataMemory
is equal to 100 MB, and a given data node is using 50 MB for data memory storage, the corresponding line in the cluster log might look like this:2006-12-24 01:18:16 [MgmSrvr] INFO -- Node 2: Data usage is 50%(1280 32K pages of total 2560)
MemReportFrequency
is not a required parameter. If used, it can be set for all cluster data nodes in the[ndbd default]
section ofconfig.ini
, and can also be set or overridden for individual data nodes in the corresponding[ndbd]
sections of the configuration file. The minimum value—which is also the default value—is 0, in which case memory reports are logged only when memory usage reaches certain percentages (80%, 90%, and 100%), as mentioned in the discussion of statistics events in Section 16.5.5.2, “MySQL Cluster Log Events”.Restart Type node Permitted Values Type numeric
When a data node is started with the
--initial
, it initializes the redo log file during Start Phase 4 (see Section 16.5.1, “Summary of MySQL Cluster Start Phases”). When very large values are set forNoOfFragmentLogFiles
,FragmentLogFileSize
, or both, this initialization can take a long time.You can force reports on the progress of this process to be logged periodically, by means of theStartupStatusReportFrequency
configuration parameter. In this case, progress is reported in the cluster log, in terms of both the number of files and the amount of space that have been initialized, as shown here:2009-06-20 16:39:23 [MgmSrvr] INFO -- Node 1: Local redo log file initialization status: #Total files: 80, Completed: 60 #Total MBytes: 20480, Completed: 15557 2009-06-20 16:39:23 [MgmSrvr] INFO -- Node 2: Local redo log file initialization status: #Total files: 80, Completed: 60 #Total MBytes: 20480, Completed: 15570
These reports are logged each
StartupStatusReportFrequency
seconds during Start Pahe 4. IfStartupStatusReportFrequency
is 0 (the default), then reports are written to the cluster log only when at the beginning and at the completion of the redo log file initialization process.
Debugging Parameters.
In MySQL Cluster NDB 7.2, it is possible to cause logging of
traces for events generated by creating and dropping tables
using DictTrace
. This
parameter is useful only in debugging NDB kernel code.
DictTrace
takes an
integer value; currently, 0 (default - no logging) and 1
(logging enabled) are the only supported values.
Backup parameters.
The [ndbd]
parameters discussed in this
section define memory buffers set aside for execution of
online backups.
Restart Type node Permitted Values Type numeric
Default 2M
Range 0 .. 4G
In creating a backup, there are two buffers used for sending data to the disk. The backup data buffer is used to fill in data recorded by scanning a node's tables. Once this buffer has been filled to the level specified as
BackupWriteSize
, the pages are sent to disk. While flushing data to disk, the backup process can continue filling this buffer until it runs out of space. When this happens, the backup process pauses the scan and waits until some disk writes have completed freed up memory so that scanning may continue.The default value for this parameter is 16MB.
Restart Type node Permitted Values Type numeric
Default 2M
Range 0 .. 4G
The backup log buffer fulfills a role similar to that played by the backup data buffer, except that it is used for generating a log of all table writes made during execution of the backup. The same principles apply for writing these pages as with the backup data buffer, except that when there is no more space in the backup log buffer, the backup fails. For that reason, the size of the backup log buffer must be large enough to handle the load caused by write activities while the backup is being made. See Section 16.5.3.3, “Configuration for MySQL Cluster Backups”.
The default value for this parameter should be sufficient for most applications. In fact, it is more likely for a backup failure to be caused by insufficient disk write speed than it is for the backup log buffer to become full. If the disk subsystem is not configured for the write load caused by applications, the cluster is unlikely to be able to perform the desired operations.
It is preferable to configure cluster nodes in such a manner that the processor becomes the bottleneck rather than the disks or the network connections.
The default value for this parameter is 16MB.
Restart Type node Permitted Values Type numeric
Default 4M
Range 0 .. 4G
This parameter is simply the sum of
BackupDataBufferSize
andBackupLogBufferSize
.The default valueof this parameter in MySQL Cluster NDB 7.2 is 16MB + 16MB = 32MB.
ImportantIf
BackupDataBufferSize
andBackupLogBufferSize
taken together exceed the default value forBackupMemory
, then this parameter must be set explicitly in theconfig.ini
file to their sum.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
This parameter controls how often backup status reports are issued in the management client during a backup, as well as how often such reports are written to the cluster log (provided cluster event logging is configured to permit it—see Logging and checkpointing).
BackupReportFrequency
represents the time in seconds between backup status reports.The default value is 0.
Restart Type node Permitted Values Type numeric
Default 32K
Range 2K .. 4G
This parameter specifies the default size of messages written to disk by the backup log and backup data buffers.
The default value for this parameter is 256KB.
Restart Type node Permitted Values (>= 5.5) Type numeric
Default 1M
Range 2K .. 4G
This parameter specifies the maximum size of messages written to disk by the backup log and backup data buffers.
The default value for this parameter is 1MB.
When specifying these parameters, the following relationships must hold true. Otherwise, the data node will be unable to start.
BackupDataBufferSize >= BackupWriteSize + 188KB
BackupLogBufferSize >= BackupWriteSize + 16KB
BackupMaxWriteSize >= BackupWriteSize
MySQL Cluster Realtime Performance Parameters
The [ndbd]
parameters discussed in this
section are used in scheduling and locking of threads to
specific CPUs on multiprocessor data node hosts.
To make use of these parameters, the data node process must be run as system root.
Restart Type node Permitted Values Type numeric
Default 64K
Range 0 .. 64K
When used with ndbd, this parameter (now a string) specifies the ID of the CPU assigned to handle the
NDBCLUSTER
execution thread. When used with ndbmtd, the value of this parameter is a comma-separated list of CPU IDs assigned to handle execution threads. Each CPU ID in the list should be an integer in the range 0 to 65535 (inclusive).The number of IDs specified should match the number of execution threads determined by
MaxNoOfExecutionThreads
. However, there is no guarantee that threads are assigned to CPUs in any given order when using this parameter; beginning with in MySQL Cluster NDB 7.2.5, you can obtain more finely-grained control of this type usingThreadConfig
.LockExecuteThreadToCPU
has no default value.Restart Type node Permitted Values Type numeric
Default 64K
Range 0 .. 64K
This parameter specifies the ID of the CPU assigned to handle
NDBCLUSTER
maintenance threads.The value of this parameter is an integer in the range 0 to 65535 (inclusive). In MySQL Cluster NDB 7.2, there is no default value.
Restart Type node Permitted Values Type boolean
Default false
Range ..
Setting this parameter to 1 enables real-time scheduling of
NDBCLUSTER
threads.The default is 0 (scheduling disabled).
Restart Type node Permitted Values Type numeric
Default 50
Range 0 .. 11000
This parameter specifies the time in microseconds for threads to be executed in the scheduler before being sent. Setting it to 0 minimizes the response time; to achieve higher throughput, you can increase the value at the expense of longer response times.
The default is 50 μsec, which our testing shows to increase throughput slightly in high-load cases without materially delaying requests.
Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 500
This parameter specifies the time in microseconds for threads to be executed in the scheduler before sleeping.
The default value is 0.
Restart Type Permitted Values Type numeric
Default 0
Range 0 .. 128
This parameter determines the number of threads to create when rebuilding indexes during a system or node start. It is supported only when there is more than one fragment for the table per data node (for example, when the
MAX_ROWS
option has been used withCREATE TABLE
).Setting this parameter to 0 (which is also the default value) disables multi-threaded building of ordered indexes. The maximum allowed value is 128.
This parameter is supported when using ndbd or ndbmtd.
You can enable multi-threaded builds during data node initial restarts by setting the
TwoPassInitialNodeRestartCopy
data node configuration parameter toTRUE
.Multi-threaded building of ordered indexes can be enabled for initial restarts of data nodes by setting this configuration parameter to
TRUE
, which enables two-pass copying of data during initial node restarts.You must also set
BuildIndexThreads
to a nonzero value.Version Introduced 5.5.17-ndb-7.2.3 Restart Type node Permitted Values Type string
Default [none]
This parameter is used with ndbmtd to assign threads of different types to different CPUs. Its value is a string whose format has the following syntax:
ThreadConfig :=
entry
[,entry
[,...]]entry
:=type
={param
[,param
[,...]]}type
:= ldm | main | recv | rep | ioparam
:= count=number
| cpubind=cpu_list
The curly braces (
{
...}
) surrounding the list of parameters is required, even if there is only one parameter in the list.The
type
attribute represents an NDB thread type. Permitted values and their associated thread types are described in the following list:ldm
: Local query handler (DBLQH
kernel block)main
: Data dictionary and transaction coordinator (DBDIH
andDBTC
kernel blocks)recv
: NDB kernel virtual machine (CMVMI
kernel block)rep
:SUMA
kernel blockio
: File system and other miscellaneous operations
A
param
(parameter) specifies the number of threads of the given type (count
), the CPUs to which the threads of the given type are to be bound (cpubind
), or both.Examples:
# Пример 1. ThreadConfig=ldm={count=2,cpubind=1,2},main={cpubind=12},rep={cpubind=11} # Пример 2. Threadconfig=main={cpubind=0},ldm={count=4,cpubind=1,2,5,6},io={count=2,cpubind=3,4}
Restart Type | node | ||
Permitted Values | |||
Type (linux) | boolean | ||
Default | 0 |
NDB
is extremely sensitive to
Non-Uniform Memory Access settings and multi-CPU systems due to
timeouts that it can cause. Due to this fact, and because most
MySQL Cluster users do not employ numactl,
support for NUMA is ignored by default by
ndbd when running on a Linux system. If your
Linux system provides NUMA support and you wish for data node
memory to be subject to NUMA control, you can set this parameter
equal to 0.
The Numa
configuration
parameter is supported only on Linux systems where
libnuma.so
is installed.
Disk Data Configuration Parameters. Configuration parameters affecting Disk Data behavior include the following:
Restart Type node Permitted Values Type numeric
Default 64M
Range 4M .. 1T
This determines the amount of space used for caching pages on disk, and is set in the
[ndbd]
or[ndbd default]
section of theconfig.ini
file. It is measured in bytes. Each page takes up 32 KB. This means that Cluster Disk Data storage always usesN
* 32 KB memory whereN
is some nonnegative integer.The default value for this parameter is
64M
(2000 pages of 32 KB each).You can query the
ndbinfo.diskpagebuffer
table to help determine whether the value for this parameter should be increased to minimize unnecessary disk seeks. See Section 16.5.9.4, “Thendbinfo diskpagebuffer
Table”, for more information.Restart Type node Permitted Values Type numeric
Default 20M
Range 0 .. 64T
This parameter determines the amount of memory that is used for log buffers, disk operations (such as page requests and wait queues), and metadata for tablespaces, log file groups,
UNDO
files, and data files. The shared global memory pool also provides memory used for satisfying the memory requirements of theINITIAL_SIZE
andUNDO_BUFFER_SIZE
options used withCREATE LOGFILE GROUP
andALTER LOGFILE GROUP
statements, including any default value implied for these options by the setting of theInitialLogFileGroup
data node configuration parameter.SharedGlobalMemory
can be set in the[ndbd]
or[ndbd default]
section of theconfig.ini
configuration file, and is measured in bytes.As of MySQL Cluster NDB 7.2.0, the default value is
128M
. (Previously, this was20M
.)Restart Type node Permitted Values Type numeric
Default 8
Range 0 .. 4G
Permitted Values Type numeric
Default 2
Range 0 .. 4G
Permitted Values Type numeric
Default 2
Range 0 .. 4G
This parameter determines the number of unbound threads used for Disk Data file access. Before
DiskIOThreadPool
was introduced, exactly one thread was spawned for each Disk Data file, which could lead to performance issues, particularly when using very large data files. WithDiskIOThreadPool
, you can—for example—access a single large data file using several threads working in parallel.Currently, this parameter applies to Disk Data I/O threads only, but we plan in the future to make the number of such threads configurable for in-memory data as well.
The optimum value for this parameter depends on your hardware and configuration, and includes these factors:
Physical distribution of Disk Data files. You can obtain better performance by placing data files, undo log files, and the data node file system on separate physical disks. If you do this with some or all of these sets of files, then you can set
DiskIOThreadPool
higher to enable separate threads to handle the files on each disk.Disk performance and types. The number of threads that can be accommodated for Disk Data file handling is also dependent on the speed and throughput of the disks. Faster disks and higher throughput allow for more disk I/O threads. Our test results indicate that solid-state disk drives can handle many more disk I/O threads than conventional disks, and thus higher values for
DiskIOThreadPool
.
In MySQL Cluster NDB 7.2, the default value for this parameter is 2.
Disk Data file system parameters. The parameters in the following list make it possible to place MySQL Cluster Disk Data files in specific directories without the need for using symbolic links.
Restart Type initial, node Permitted Values Type file name
Default FileSystemPath
Range ..
If this parameter is specified, then MySQL Cluster Disk Data data files and undo log files are placed in the indicated directory. This can be overridden for data files, undo log files, or both, by specifying values for
FileSystemPathDataFiles
,FileSystemPathUndoFiles
, or both, as explained for these parameters. It can also be overridden for data files by specifying a path in theADD DATAFILE
clause of aCREATE TABLESPACE
orALTER TABLESPACE
statement, and for undo log files by specifying a path in theADD UNDOFILE
clause of aCREATE LOGFILE GROUP
orALTER LOGFILE GROUP
statement. IfFileSystemPathDD
is not specified, thenFileSystemPath
is used.If a
FileSystemPathDD
directory is specified for a given data node (including the case where the parameter is specified in the[ndbd default]
section of theconfig.ini
file), then starting that data node with--initial
causes all files in the directory to be deleted.Restart Type initial, node Permitted Values Type file name
Default FileSystemPathDD
Range ..
If this parameter is specified, then MySQL Cluster Disk Data data files are placed in the indicated directory. This overrides any value set for
FileSystemPathDD
. This parameter can be overridden for a given data file by specifying a path in theADD DATAFILE
clause of aCREATE TABLESPACE
orALTER TABLESPACE
statement used to create that data file. IfFileSystemPathDataFiles
is not specified, thenFileSystemPathDD
is used (orFileSystemPath
, ifFileSystemPathDD
has also not been set).If a
FileSystemPathDataFiles
directory is specified for a given data node (including the case where the parameter is specified in the[ndbd default]
section of theconfig.ini
file), then starting that data node with--initial
causes all files in the directory to be deleted.Restart Type initial, node Permitted Values Type file name
Default FileSystemPathDD
Range ..
If this parameter is specified, then MySQL Cluster Disk Data undo log files are placed in the indicated directory. This overrides any value set for
FileSystemPathDD
. This parameter can be overridden for a given data file by specifying a path in theADD UNDO
clause of aCREATE LOGFILE GROUP
orCREATE LOGFILE GROUP
statement used to create that data file. IfFileSystemPathUndoFiles
is not specified, thenFileSystemPathDD
is used (orFileSystemPath
, ifFileSystemPathDD
has also not been set).If a
FileSystemPathUndoFiles
directory is specified for a given data node (including the case where the parameter is specified in the[ndbd default]
section of theconfig.ini
file), then starting that data node with--initial
causes all files in the directory to be deleted.
For more information, see Section 16.5.11.1, “MySQL Cluster Disk Data Objects”.
Disk Data object creation parameters. The next two parameters enable you—when starting the cluster for the first time—to cause a Disk Data log file group, tablespace, or both, to be created without the use of SQL statements.
Restart Type system Permitted Values Type string
Default [see documentation]
Range ..
This parameter can be used to specify a log file group that is created when performing an initial start of the cluster.
InitialLogFileGroup
is specified as shown here:InitialLogFileGroup = [name=
name
;] [undo_buffer_size=size
;]file-specification-list
file-specification-list
:file-specification
[;file-specification
[; ...]]file-specification
:filename
:size
The
name
of the log file group is optional and defaults toDEFAULT-LG
. Theundo_buffer_size
is also optional; if omitted, it defaults to64M
. Eachfile-specification
corresponds to an undo log file, and at least one must be specified in thefile-specification-list
. Undo log files are placed according to any values that have been set forFileSystemPath
,FileSystemPathDD
, andFileSystemPathUndoFiles
, just as if they had been created as the result of aCREATE LOGFILE GROUP
orALTER LOGFILE GROUP
statement.Consider the following:
InitialLogFileGroup = name=LG1; undo_buffer_size=128M; undo1.log:250M; undo2.log:150M
This is equivalent to the following SQL statements:
CREATE LOGFILE GROUP LG1 ADD UNDOFILE 'undo1.log' INITIAL_SIZE 250M UNDO_BUFFER_SIZE 128M ENGINE NDBCLUSTER; ALTER LOGFILE GROUP LG1 ADD UNDOFILE 'undo2.log' INITIAL_SIZE 150M ENGINE NDBCLUSTER;
This logfile group is created when the data nodes are started with
--initial
.Resources for the initial log file group are taken from the global memory pool whose size is determined by the value of the
SharedGlobalMemory
data node configuration parameter; if this parameter is set too low and the values set inInitialLogFileGroup
for the logfile group's initial size or undo buffer size are too high, the cluster may fail to create the default log file group when starting, or fail to start altogether.This parameter, if used, should always be set in the
[ndbd default]
section of theconfig.ini
file. The behavior of a MySQL Cluster when different values are set on different data nodes is not defined.Restart Type system Permitted Values Type string
Default [see documentation]
Range ..
This parameter can be used to specify a MySQL Cluster Disk Data tablespace that is created when performing an initial start of the cluster.
InitialTablespace
is specified as shown here:InitialTablespace = [name=
name
;] [extent_size=size
;]file-specification-list
The
name
of the tablespace is optional and defaults toDEFAULT-TS
. Theextent_size
is also optional; it defaults to1M
. Thefile-specification-list
uses the same syntax as shown with theInitialLogfileGroup
parameter, the only difference being that eachfile-specification
used withInitialTablespace
corresponds to a data file. At least one must be specified in thefile-specification-list
. Data files are placed according to any values that have been set forFileSystemPath
,FileSystemPathDD
, andFileSystemPathDataFiles
, just as if they had been created as the result of aCREATE TABLESPACE
orALTER TABLESPACE
statement.For example, consider the following line specifying
InitialTablespace
in the[ndbd default]
section of theconfig.ini
file (as withInitialLogfileGroup
, this parameter should always be set in the[ndbd default]
section, as the behavior of a MySQL Cluster when different values are set on different data nodes is not defined):InitialTablespace = name=TS1; extent_size=8M; data1.dat:2G; data2.dat:4G
This is equivalent to the following SQL statements:
CREATE TABLESPACE TS1 ADD DATAFILE 'data1.dat' EXTENT_SIZE 8M INITIAL_SIZE 2G ENGINE NDBCLUSTER; ALTER TABLESPACE TS1 ADD UNDOFILE 'data2.dat' INITIAL_SIZE 4G ENGINE NDBCLUSTER;
This tablespace is created when the data nodes are started with
--initial
, and can be used whenever creating MySQL Cluster Disk Data tables thereafter.
Disk Data and GCP Stop errors.
Ошибки encountered when using Disk Data tables such as
Node nodeid
killed this
node because GCP stop was detected (error 2303)
are often referred to as “GCP stop errors”. Such
errors occur when the redo log is not flushed to disk quickly
enough; this is usually due to slow disks and insufficient
disk throughput.
You can help prevent these errors from occurring by using faster
disks, and by placing Disk Data files on a separate disk from
the data node file system. Reducing the value of
TimeBetweenGlobalCheckpoints
tends to decrease the amount of data to be written for each
global checkpoint, and so may provide some protection against
redo log buffer overflows when trying to write a global
checkpoint; however, reducing this value also permits less time
in which to write the GCP, so this must be done with caution.
In addition to the considerations given for
DiskPageBufferMemory
as
explained previously, it is also very important that the
DiskIOThreadPool
configuration parameter be set correctly; having
DiskIOThreadPool
set too
high is very likely to cause GCP stop errors (Bug #37227).
GCP stops can be caused by save or commit timeouts; the
TimeBetweenEpochsTimeout
data node configuration parameter determines the timeout for
commits. However, it is possible to disable both types of
timeouts by setting this parameter to 0.
Parameters for configuring send buffer memory allocation.
Send buffer memory is allocated dynamically from a memory pool
shared between all transporters, which means that the size of
the send buffer can be adjusted as necessary. (Previously, the
NDB kernel used a fixed-size send buffer for every node in the
cluster, which was allocated when the node started and could
not be changed while the node was running.) The
TotalSendBufferMemory
and OverLoadLimit
data
node configuration parameters permit the setting of limits on
this memory allocation. For more information about the use of
these parameters (as well as
SendBufferMemory
), see
Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”.
This parameter specifies the amount of transporter send buffer memory to allocate in addition to any set using
TotalSendBufferMemory
,SendBufferMemory
, or both.This parameter was added in MySQL Cluster NDB 7.2.5. (Bug #11760629, Bug #53053)
This parameter is available beginning with MySQL Cluster NDB 6.4.0. It is used to determine the total amount of memory to allocate on this node for shared send buffer memory among all configured transporters.
Prior to MySQL Cluster NDB 7.2.5, this parameter did not work correctly with ndbmtd. (Bug #13633845)
If this parameter is set, its minimum permitted value is 256KB; the maxmimum is 4294967039.
This parameter is present in
NDBCLUSTER
source code beginning with MySQL Cluster NDB 6.4.0. However, it is not currently enabled.As of MySQL Cluster NDB 7.2.5, this parameter is deprecated, and is subject to removal in a future release of MySQL Cluster (Bug #11760629, Bug #53053).
For more detailed information about the behavior and use of
TotalSendBufferMemory
and about configuring send buffer memory parameters in MySQL
Cluster, see
Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”.
See also Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”.
Redo log over-commit handling.
It is possible to control a data node's handling of
operations when too much time is taken flushing redo logs to
disk. This occurs when a given redo log flush takes longer
than
RedoOverCommitLimit
seconds, more than
RedoOverCommitCounter
times, causing any pending transactions to be aborted. When
this happens, the API node that sent the transaction can
handle the operations that should have been committed either
by queuing the operations and re-trying them, or by aborting
them, as determined by
DefaultOperationRedoProblemAction
.
The data node configuration parameters for setting the timeout
and number of times it may be exceeded before the API node
takes this action are described in the following list:
Restart Type node Permitted Values Type numeric
Default 3
Range 0 .. 4G
When
RedoOverCommitLimit
is exceeded when trying to write a given redo log to disk this many times or more, any transactions that were not committed as a result are aborted, and an API node where any of these transactions originated handles the operations making up those transactions according to its value forDefaultOperationRedoProblemAction
(by either queuing the operations to be re-tried, or aborting them).RedoOverCommitCounter
defaults to 3. Set it to 0 to disable the limit.Restart Type node Permitted Values Type numeric
Default 20
Range 0 .. 4G
This parameter sets an upper limit in seconds for trying to write a given redo log to disk before timing out. The number of times the data node tries to flush this redo log, but takes longer than
RedoOverCommitLimit
, is kept and compared withRedoOverCommitCounter
, and when flushing takes too long more times than the value of that parameter, any transactions that were not committed as a result of the flush timeout are aborted. When this occurs, the API node where any of these transactions originated handles the operations making up those transactions according to itsDefaultOperationRedoProblemAction
setting (it either queues the operations to be re-tried, or aborts them).By default,
RedoOverCommitLimit
is 20 seconds. Set to 0 to disable checking for redo log flush timeouts. This parameter was added in MySQL Cluster NDB 7.1.10.
Controlling restart attempts.
It is possible to exercise finely-grained control over restart
attempts by data nodes when they fail to start using the
MaxStartFailRetries
and
StartFailRetryDelay
data node configuration parameters.
MaxStartFailRetries
limits the total number of retries made before giving up on
starting the data node,
StartFailRetryDelay
sets
the number of seconds between retry attempts. These parameters
are described in more detail in the next few paragraphs.
Restart Type | node | ||
Permitted Values | |||
Type | numeric | ||
Default | 0 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 0 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 0 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 0 | ||
Range | 0 .. 4G |
Use this parameter to set the number of seconds between restart attempts by the data node in the event on failure on startup. The default is 0 (no delay).
This parameter is ignored unless
StartOnError
is equal to 0.
Restart Type | node | ||
Permitted Values | |||
Type | numeric | ||
Default | 3 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 3 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 3 | ||
Range | 0 .. 4G | ||
Permitted Values | |||
Type | numeric | ||
Default | 3 | ||
Range | 0 .. 4G |
Use this parameter to limit the number restart attempts made by the data node in the event that it fails on startup. The default is 3 attempts.
This parameter is ignored unless
StopOnError
is equal
to 0.
The [mysqld]
and [api]
sections in the config.ini
file define the
behavior of the MySQL servers (SQL nodes) and other applications
(API nodes) used to access cluster data. None of the parameters
shown is required. If no computer or host name is provided, any
host can use this SQL or API node.
Generally speaking, a [mysqld]
section is
used to indicate a MySQL server providing an SQL interface to
the cluster, and an [api]
section is used for
applications other than mysqld processes
accessing cluster data, but the two designations are actually
synonomous; you can, for instance, list parameters for a MySQL
server acting as an SQL node in an [api]
section.
For a discussion of MySQL server options for MySQL Cluster, see Section 16.3.4.2, “MySQL Server Options for MySQL Cluster”; for information about MySQL server system variables relating to MySQL Cluster, see Section 16.3.4.3, “MySQL Cluster System Variables”.
Deprecated 5.1.51-ndb-7.1.9 Restart Type node Permitted Values Type numeric
Default Range 1 .. 63
The
Id
is an integer value used to identify the node in all cluster internal messages. The permitted range of values is 1 to 255 inclusive. This value must be unique for each node in the cluster, regardless of the type of node.ЗамечаниеData node IDs must be less than 49, regardless of the MySQL Cluster version used. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for API nodes (and management nodes) to values greater than 48.
NodeId
is the preferred parameter name to use when identifying API nodes. (Id
continues to be supported for backward compatibility, but is now deprecated and generates a warning when used. It is also subject to future removal.)Restart Type node Permitted Values Type numeric
Default Range 1 .. 63
The
NodeId
is an integer value used to identify the node in all cluster internal messages. The permitted range of values is 1 to 255 inclusive. This value must be unique for each node in the cluster, regardless of the type of node.ЗамечаниеData node IDs must be less than 49, regardless of the MySQL Cluster version used. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for API nodes (and management nodes) to values greater than 48.
NodeId
is the preferred parameter name to use when identifying management nodes in MySQL Cluster NDB 7.2 and later. Previously,Id
was used for this purpose and this continues to be supported for backward compatibility.Id
is now deprecated and generates a warning when used; it is subject to removal in a future release of MySQL Cluster.Restart Type system Permitted Values Type string
Default Range ..
This refers to the
Id
set for one of the computers (hosts) defined in a[computer]
section of the configuration file.Restart Type system Permitted Values Type string
Default Range ..
Specifying this parameter defines the hostname of the computer on which the SQL node (API node) is to reside. To specify a hostname, either this parameter or
ExecuteOnComputer
is required.If no
HostName
orExecuteOnComputer
is specified in a given[mysql]
or[api]
section of theconfig.ini
file, then an SQL or API node may connect using the corresponding “slot” from any host which can establish a network connection to the management server host machine. This differs from the default behavior for data nodes, wherelocalhost
is assumed forHostName
unless otherwise specified.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 2
This parameter defines which nodes can act as arbitrators. Both MGM nodes and SQL nodes can be arbitrators. A value of 0 means that the given node is never used as an arbitrator, a value of 1 gives the node high priority as an arbitrator, and a value of 2 gives it low priority. A normal configuration uses the management server as arbitrator, setting its
ArbitrationRank
to 1 (the default for management nodes) and those for all SQL nodes to 0 (the default for SQL nodes).By setting
ArbitrationRank
to 0 on all management and SQL nodes, you can disable arbitration completely. You can also control arbitration by overriding this parameter; to do so, set theArbitration
parameter in the[ndbd default]
section of theconfig.ini
global configuration file.Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
Setting this parameter to any other value than 0 (the default) means that responses by the arbitrator to arbitration requests will be delayed by the stated number of milliseconds. It is usually not necessary to change this value.
Restart Type node Permitted Values Type numeric
Default 32K
Range 1024 .. 1M
For queries that are translated into full table scans or range scans on indexes, it is important for best performance to fetch records in properly sized batches. It is possible to set the proper size both in terms of number of records (
BatchSize
) and in terms of bytes (BatchByteSize
). The actual batch size is limited by both parameters.The speed at which queries are performed can vary by more than 40% depending upon how this parameter is set. In future releases, MySQL Server will make educated guesses on how to set parameters relating to batch size, based on the query type.
This parameter is measured in bytes and by default is equal to 16KB in MySQL Cluster NDB 7.2.1 and later. (Previously, the default was 32K.)
Restart Type node Permitted Values Type numeric
Default 64
Range 1 .. 992
This parameter is measured in number of records and is by default set to 256 (MySQL Cluster NDB 7.2.1 and later; previously, the default was 64). The maximum size is 992.
Use this parameter to set the scheduling policy and priority of heartbeat threads for management and API nodes. The syntax for setting this parameter is shown here:
HeartbeatThreadPriority =
policy
[,priority
]policy
: {FIFO | RR}When setting this parameter, you must specify a policy. This is one of
FIFO
(first in, first in) orRR
(round robin). This followed optionally by the priority (an integer).Restart Type node Permitted Values Type numeric
Default 256K
Range 32K .. 16M
The batch size is the size of each batch sent from each data node. Most scans are performed in parallel to protect the MySQL Server from receiving too much data from many nodes in parallel; this parameter sets a limit to the total batch size over all nodes.
The default value of this parameter is set to 256KB. Its maximum size is 16MB.
Restart Type node Permitted Values Type numeric
Default 256K
Range 0 .. 4G
This parameter is available beginning with MySQL Cluster NDB 6.4.0. It is used to determine the total amount of memory to allocate on this node for shared send buffer memory among all configured transporters.
If this parameter is set, its minimum permitted value is 256KB; the maxmimum is 4294967039. For more detailed information about the behavior and use of
TotalSendBufferMemory
and configuring send buffer memory parameters in MySQL Cluster, see Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”.Restart Type node Permitted Values Type boolean
Default false
Range false .. true
This parameter is
false
by default. This forces disconnected API nodes (including MySQL Servers acting as SQL nodes) to use a new connection to the cluster rather than attempting to re-use an existing one, as re-use of connections can cause problems when using dynamically-allocated node IDs. (Bug #45921)ЗамечаниеThis parameter can be overridden using the NDB API. For more information, see
Ndb_cluster_connection::set_auto_reconnect()
, andNdb_cluster_connection::get_auto_reconnect()
.DefaultOperationRedoProblemAction
Restart Type Permitted Values Type enumeration
Default Valid Values ABORT
QUEUE
This parameter (along with
RedoOverCommitLimit
andRedoOverCommitCounter
) controls the data node's handling of operations when too much time is taken flushing redo logs to disk. This occurs when a given redo log flush takes longer thanRedoOverCommitLimit
seconds, more thanRedoOverCommitCounter
times, causing any pending transactions to be aborted.When this happens, the node can respond in either of two ways, according to the value of
DefaultOperationRedoProblemAction
, listed here:ABORT
: Any pending operations from aborted transactions are also aborted.QUEUE
: Pending operations from transactions that were aborted are queued up to be re-tried.
You can obtain some information from a MySQL server running as a
Cluster SQL node using SHOW
STATUS
in the mysql
client, as
shown here:
mysql> SHOW STATUS LIKE 'ndb%';
+-----------------------------+---------------+
| Variable_name | Value |
+-----------------------------+---------------+
| Ndb_cluster_node_id | 5 |
| Ndb_config_from_host | 192.168.0.112 |
| Ndb_config_from_port | 1186 |
| Ndb_number_of_storage_nodes | 4 |
+-----------------------------+---------------+
4 rows in set (0.02 sec)
For information about these Cluster system status variables, see Section 5.1.5, “Server Status Variables”.
To add new SQL or API nodes to the configuration of a running
MySQL Cluster, it is necessary to perform a rolling restart of
all cluster nodes after adding new [mysqld]
or [api]
sections to the
config.ini
file (or files, if you are
using more than one management server). This must be done
before the new SQL or API nodes can connect to the cluster.
It is not necessary to perform any restart of the cluster if new SQL or API nodes can employ previously unused API slots in the cluster configuration to connect to the cluster.
TCP/IP is the default transport mechanism for all connections between nodes in a MySQL Cluster. Normally it is not necessary to define TCP/IP connections; MySQL Cluster automatically sets up such connections for all data nodes, management nodes, and SQL or API nodes.
For an exception to this rule, see Section 16.3.2.9, “MySQL Cluster TCP/IP Connections Using Direct Connections”.
To override the default connection parameters, it is necessary
to define a connection using one or more
[tcp]
sections in the
config.ini
file. Each
[tcp]
section explicitly defines a TCP/IP
connection between two MySQL Cluster nodes, and must contain at
a minimum the parameters
NodeId1
and
NodeId2
, as well as any
connection parameters to override.
It is also possible to change the default values for these
parameters by setting them in the [tcp
default]
section.
Any [tcp]
sections in the
config.ini
file should be listed
last, following all other sections in the
file. However, this is not required for a [tcp
default]
section. This requirement is a known issue
with the way in which the config.ini
file
is read by the MySQL Cluster management server.
Connection parameters which can be set in
[tcp]
and [tcp default]
sections of the config.ini
file are listed
here:
Restart Type node Permitted Values Type numeric
Default Range ..
Restart Type node Permitted Values Type numeric
Default Range ..
To identify a connection between two nodes it is necessary to provide their node IDs in the
[tcp]
section of the configuration file. These are the same uniqueNodeId
(orId
) values for each of these nodes as described in Section 16.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.Restart Type node Permitted Values Type string
Default Range ..
Restart Type node Permitted Values Type string
Default Range ..
The
HostName1
andHostName2
parameters can be used to specify specific network interfaces to be used for a given TCP connection between two nodes. The values used for these parameters can be hostnames or IP addresses.This parameter can be used to determine the amount of unsent data that must be present in the send buffer before the connection is considered overloaded. See Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”, for more information.
In some older releases, the effective value of this parameter was limited by the size of
SendBufferMemory
; in MySQL Cluster NDB 7.2, the actual value forOverloadLimit
(up to the stated maximum of 4G) is used instead.Restart Type node Permitted Values Type numeric
Default 256K
Range 64K .. 4G
TCP transporters use a buffer to store all messages before performing the send call to the operating system. When this buffer reaches 64KB its contents are sent; these are also sent when a round of messages have been executed. To handle temporary overload situations it is also possible to define a bigger send buffer.
If this parameter is set explicitly, then the memory is not dedicated to each transporter; instead, the value used denotes the hard limit for how much memory (out of the total available memory—that is,
TotalSendBufferMemory
) that may be used by a single transporter. For more information about configuring dynamic transporter send buffer memory allocation in MySQL Cluster, see Section 16.3.2.12, “Configuring MySQL Cluster Send Buffer Parameters”.The default size of the send buffer in MySQL Cluster NDB 7.2 is 2MB, which is the size recommended in most situations. The minimum size is 64 KB; the theoretical maximum is 4 GB.
Restart Type node Permitted Values Type boolean
Default false (debug builds: true)
Range ..
To be able to retrace a distributed message datagram, it is necessary to identify each message. When this parameter is set to
Y
, message IDs are transported over the network. This feature is disabled by default in production builds, and enabled in-debug
builds.Restart Type node Permitted Values Type boolean
Default false
Range ..
This parameter is a boolean parameter (enabled by setting it to
Y
or1
, disabled by setting it toN
or0
). It is disabled by default. When it is enabled, checksums for all messages are calculated before they placed in the send buffer. This feature ensures that messages are not corrupted while waiting in the send buffer, or by the transport mechanism.This formerly specified the port number to be used for listening for connections from other nodes. This parameter should no longer be used; use the
ServerPort
data node configuration parameter for this purpose instead.Restart Type node Permitted Values Type numeric
Default 64K
Range 16K .. 4G
Specifies the size of the buffer used when receiving data from the TCP/IP socket.
The default value of this parameter is 2MB. The minimum possible value is 16KB; the theoretical maximum is 4GB.
Setting up a cluster using direct connections between data nodes
requires specifying explicitly the crossover IP addresses of the
data nodes so connected in the [tcp]
section
of the cluster config.ini
file.
In the following example, we envision a cluster with at least
four hosts, one each for a management server, an SQL node, and
two data nodes. The cluster as a whole resides on the
172.23.72.*
subnet of a LAN. In addition to
the usual network connections, the two data nodes are connected
directly using a standard crossover cable, and communicate with
one another directly using IP addresses in the
1.1.0.*
address range as shown:
# Management Server [ndb_mgmd] Id=1 HostName=172.23.72.20 # SQL Node [mysqld] Id=2 HostName=172.23.72.21 # Data Nodes [ndbd] Id=3 HostName=172.23.72.22 [ndbd] Id=4 HostName=172.23.72.23 # TCP/IP Connections [tcp] NodeId1=3 NodeId2=4 HostName1=1.1.0.1 HostName2=1.1.0.2
The HostName1
and
HostName2
parameters are
used only when specifying direct connections.
The use of direct TCP connections between data nodes can improve the cluster's overall efficiency by enabling the data nodes to bypass an Ethernet device such as a switch, hub, or router, thus cutting down on the cluster's latency. It is important to note that to take the best advantage of direct connections in this fashion with more than two data nodes, you must have a direct connection between each data node and every other data node in the same node group.
MySQL Cluster attempts to use the shared memory transporter and
configure it automatically where possible.
[shm]
sections in the
config.ini
file explicitly define
shared-memory connections between nodes in the cluster. When
explicitly defining shared memory as the connection method, it
is necessary to define at least
NodeId1
,
NodeId2
, and
ShmKey
. All other
parameters have default values that should work well in most
cases.
SHM functionality is considered experimental only. It is not officially supported in any current MySQL Cluster release, and testing results indicate that SHM performance is not appreciably greater than when using TCP/IP for the transporter.
For these reasons, you must determine for yourself or by using our free resources (forums, mailing lists) whether SHM can be made to work correctly in your specific case.
Restart Type node Permitted Values Type numeric
Default Range ..
Restart Type node Permitted Values Type numeric
Default Range ..
To identify a connection between two nodes it is necessary to provide node identifiers for each of them, as
NodeId1
andNodeId2
.Restart Type node Permitted Values Type string
Default Range ..
Restart Type node Permitted Values Type string
Default Range ..
The
HostName1
andHostName2
parameters can be used to specify specific network interfaces to be used for a given SHM connection between two nodes. The values used for these parameters can be hostnames or IP addresses.Restart Type node Permitted Values Type numeric
Default Range 0 .. 4G
When setting up shared memory segments, a node ID, expressed as an integer, is used to identify uniquely the shared memory segment to use for the communication. There is no default value.
Restart Type node Permitted Values Type numeric
Default 1M
Range 64K .. 4G
Each SHM connection has a shared memory segment where messages between nodes are placed by the sender and read by the reader. The size of this segment is defined by
ShmSize
. The default value is 1MB.Restart Type node Permitted Values Type boolean
Default false
Range ..
To retrace the path of a distributed message, it is necessary to provide each message with a unique identifier. Setting this parameter to
Y
causes these message IDs to be transported over the network as well. This feature is disabled by default in production builds, and enabled in-debug
builds.Restart Type node Permitted Values Type boolean
Default true
Range ..
This parameter is a boolean (
Y
/N
) parameter which is disabled by default. When it is enabled, checksums for all messages are calculated before being placed in the send buffer.This feature prevents messages from being corrupted while waiting in the send buffer. It also serves as a check against data being corrupted during transport.
Restart Type node Permitted Values Type numeric
Default Range 0 .. 4G
When using the shared memory transporter, a process sends an operating system signal to the other process when there is new data available in the shared memory. Should that signal conflict with with an existing signal, this parameter can be used to change it. This is a possibility when using SHM due to the fact that different operating systems use different signal numbers.
The default value of
SigNum
is 0; therefore, it must be set to avoid errors in the cluster log when using the shared memory transporter. Typically, this parameter is set to 10 in the[shm default]
section of theconfig.ini
file.
[sci]
sections in the
config.ini
file explicitly define SCI
(Scalable Coherent Interface) connections between cluster nodes.
Using SCI transporters in MySQL Cluster is supported only when
the MySQL binaries are built using
--with-ndb-sci=
.
The /your/path/to/SCI
path
should point to a directory
that contains at a minimum lib
and
include
directories containing SISCI
libraries and header files. (See
Section 16.3.5, “Using High-Speed Interconnects with MySQL Cluster” for more
information about SCI.)
In addition, SCI requires specialized hardware.
It is strongly recommended to use SCI Transporters only for communication between ndbd processes. Note also that using SCI Transporters means that the ndbd processes never sleep. For this reason, SCI Transporters should be used only on machines having at least two CPUs dedicated for use by ndbd processes. There should be at least one CPU per ndbd process, with at least one CPU left in reserve to handle operating system activities.
Restart Type node Permitted Values Type numeric
Default Range ..
Restart Type node Permitted Values Type numeric
Default Range ..
To identify a connection between two nodes it is necessary to provide node identifiers for each of them, as
NodeId1
andNodeId2
.Restart Type node Permitted Values Type numeric
Default Range 0 .. 4G
This identifies the SCI node ID on the first Cluster node (identified by
NodeId1
).Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
It is possible to set up SCI Transporters for failover between two SCI cards which then should use separate networks between the nodes. This identifies the node ID and the second SCI card to be used on the first node.
Restart Type node Permitted Values Type numeric
Default Range 0 .. 4G
This identifies the SCI node ID on the second Cluster node (identified by
NodeId2
).Restart Type node Permitted Values Type numeric
Default 0
Range 0 .. 4G
When using two SCI cards to provide failover, this parameter identifies the second SCI card to be used on the second node.
Restart Type node Permitted Values Type string
Default Range ..
Restart Type node Permitted Values Type string
Default Range ..
The
HostName1
andHostName2
parameters can be used to specify specific network interfaces to be used for a given SCI connection between two nodes. The values used for these parameters can be hostnames or IP addresses.Restart Type node Permitted Values Type numeric
Default 10M
Range 64K .. 4G
Each SCI transporter has a shared memory segment used for communication between the two nodes. Setting the size of this segment to the default value of 1MB should be sufficient for most applications. Using a smaller value can lead to problems when performing many parallel inserts; if the shared buffer is too small, this can also result in a crash of the ndbd process.
Restart Type node Permitted Values Type numeric
Default 8K
Range 128 .. 32K
A small buffer in front of the SCI media stores messages before transmitting them over the SCI network. By default, this is set to 8KB. Our benchmarks show that performance is best at 64KB but 16KB reaches within a few percent of this, and there was little if any advantage to increasing it beyond 8KB.
Restart Type node Permitted Values Type boolean
Default true
Range ..
To trace a distributed message it is necessary to identify each message uniquely. When this parameter is set to
Y
, message IDs are transported over the network. This feature is disabled by default in production builds, and enabled in-debug
builds.Restart Type node Permitted Values Type boolean
Default false
Range ..
This parameter is a boolean value, and is disabled by default. When
Checksum
is enabled, checksums are calculated for all messages before they are placed in the send buffer. This feature prevents messages from being corrupted while waiting in the send buffer. It also serves as a check against data being corrupted during transport.
Formerly, the NDB kernel employed a send buffer whose size was fixed at 2MB for each node in the cluster, this buffer being allocated when the node started. Because the size of this buffer could not be changed after the cluster was started, it was necessary to make it large enough in advance to accommodate the maximum possible load on any transporter socket. However, this was an inefficient use of memory, since much of it often went unused, and could result in large amounts of resources being wasted when scaling up to many API nodes.
This problem was eventually solved (in MySQL Cluster NDB 7.0) by employing a unified send buffer whose memory is allocated dynamically from a pool shared by all transporters. This means that the size of the send buffer can be adjusted as necessary. Configuration of the unified send buffer can accomplished by setting the following parameters:
TotalSendBufferMemory
. This parameter can be set for all types of MySQL Cluster nodes—that is, it can be set in the[ndbd]
,[mgm]
, and[api]
(or[mysql]
) sections of theconfig.ini
file. It represents the total amount of memory (in bytes) to be allocated by each node for which it is set for use among all configured transporters. If set, its minimum is 256KB; the maximum is 4294967039.To be backward-compatible with existing configurations, this parameter takes as its default value the sum of the maximum send buffer sizes of all configured transporters, plus an additional 32KB (one page) per transporter. The maximum depends on the type of transporter, as shown in the following table:
Transporter Maxmimum Send Buffer Size (bytes) TCP SendBufferMemory
(default = 2M)SCI SendLimit
(default = 8K) plus 16KSHM 20K This enables existing configurations to function in close to the same way as they did with MySQL Cluster NDB 6.3 and earlier, with the same amount of memory and send buffer space available to each transporter. However, memory that is unused by one transporter is not available to other transporters.
OverloadLimit
. This parameter is used in theconfig.ini
file[tcp]
section, and denotes the amount of unsent data (in bytes) that must be present in the send buffer before the connection is considered overloaded. When such an overload condition occurs, transactions that affect the overloaded connection fail with NDB API Error 1218 (Send Buffers overloaded in NDB kernel) until the overload status passes. The default value is 0, in which case the effective overload limit is calculated asSendBufferMemory * 0.8
for a given connection. The maximum value for this parameter is 4G.SendBufferMemory
. This value denotes a hard limit for the amount of memory that may be used by a single transporter out of the entire pool specified byTotalSendBufferMemory
. However, the sum ofSendBufferMemory
for all configured transporters may be greater than theTotalSendBufferMemory
that is set for a given node. This is a way to save memory when many nodes are in use, as long as the maximum amount of memory is never required by all transporters at the same time.
The next four sections provide summary tables of MySQL Cluster
configuration parameters used in the
config.ini
file to govern the cluster's
functioning. Each table lists the parameters for one of the
Cluster node process types (ndbd,
ndb_mgmd, and mysqld), and
includes the parameter's type as well as its default, mimimum, and
maximum values as applicable.
These tables also indicate what type of restart is required (node
restart or system restart)—and whether the restart must be
done with --initial
—to change the value of
a given configuration parameter.
When performing a node restart or an initial node restart, all of
the cluster's data nodes must be restarted in turn (also referred
to as a rolling restart). It is possible to
update cluster configuration parameters marked as
node
online—that is, without shutting
down the cluster—in this fashion. An initial node restart
requires restarting each ndbd process with the
--initial
option.
A system restart requires a complete shutdown and restart of the entire cluster. An initial system restart requires taking a backup of the cluster, wiping the cluster file system after shutdown, and then restoring from the backup following the restart.
In any cluster restart, all of the cluster's management servers must be restarted for them to read the updated configuration parameter values.
Values for numeric cluster parameters can generally be increased without any problems, although it is advisable to do so progressively, making such adjustments in relatively small increments. Many of these can be increased online, using a rolling restart.
However, decreasing the values of such parameters—whether
this is done using a node restart, node initial restart, or even
a complete system restart of the cluster—is not to be
undertaken lightly; it is recommended that you do so only after
careful planning and testing. This is especially true with
regard to those parameters that relate to memory usage and disk
space, such as
MaxNoOfTables
,
MaxNoOfOrderedIndexes
,
and
MaxNoOfUniqueHashIndexes
.
In addition, it is the generally the case that configuration
parameters relating to memory and disk usage can be raised using
a simple node restart, but they require an initial node restart
to be lowered.
Because some of these parameters can be used for configuring more than one type of cluster node, they may appear in more than one of the tables.
4294967039
often appears as a maximum value
in these tables. This value is defined in the
NDBCLUSTER
sources as
MAX_INT_RNIL
and is equal to
0xFFFFFEFF
, or
232 –
28 – 1
.
The summary table in this section provides information about
parameters used in the [ndbd]
or
[ndbd default]
sections of a
config.ini
file for configuring MySQL
Cluster data nodes. For detailed descriptions and other
additional information about each of these parameters, see
Section 16.3.2.6, “Defining MySQL Cluster Data Nodes”.
These parameters also apply to ndbmtd, the multi-threaded version of ndbd. For more information, see Section 16.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.
Restart types. Changes in MySQL Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary table as follows:
N
—Node restart: The parameter can be updated using a rolling restart (see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”).S
—System restart: The cluster must be shut down completely, then restarted, to effect a change in this parameter.I
—Initial restart: Data nodes must be restarted using the--initial
option.
For more information about restart types, see Section 16.3.3, “Overview of MySQL Cluster Configuration Parameters”.
MySQL Cluster NDB 7.2 supports the addition of new data node groups online, to a running cluster. For more information, see Section 16.5.12, “Adding MySQL Cluster Data Nodes Online”.
Table 16.1. Data Node Configuration Parameters
The summary table in this section provides information about
parameters used in the [ndb_mgmd]
or
[mgm]
sections of a
config.ini
file for configuring MySQL
Cluster management nodes. For detailed descriptions and other
additional information about each of these parameters, see
Section 16.3.2.5, “Defining a MySQL Cluster Management Server”.
Restart types. Changes in MySQL Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary table as follows:
N
—Node restart: The parameter can be updated using a rolling restart (see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”).S
—System restart: The cluster must be shut down completely, then restarted, to effect a change in this parameter.I
—Initial restart: Data nodes must be restarted using the--initial
option.
For more information about restart types, see Section 16.3.3, “Overview of MySQL Cluster Configuration Parameters”.
Table 16.2. Management Node Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
ArbitrationDelay | milliseconds | 4G | N | ||
ArbitrationRank | 0-2 | 1 | 2 | N | |
DataDir | path | . | N | ||
ExecuteOnComputer | name | S | |||
HeartbeatThreadPriority | none | ||||
HostName | name or IP address | S | |||
Id | unsigned | 1 | 63 | N | |
LogDestination | {CONSOLE|SYSLOG|FILE} | FILE:filename=ndb_nodeid_cluster.log,maxsize=1000000,maxfiles=6 | N | ||
MaxNoOfSavedEvents | unsigned | 100 | 4G | N | |
NodeId | unsigned | 1 | 63 | N | |
PortNumber | unsigned | 1186 | 64K | N | |
PortNumberStats | unsigned | 64K | N | ||
wan | false | N |
After making changes in a management node's configuration, it is necessary to perform a rolling restart of the cluster for the new configuration to take effect. See Section 16.3.2.5, “Defining a MySQL Cluster Management Server”, for more information.
To add new management servers to a running MySQL Cluster, it
is also necessary perform a rolling restart of all cluster
nodes after modifying any existing
config.ini
files. For more information
about issues arising when using multiple management nodes, see
Section 16.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
The summary table in this section provides information about
parameters used in the [mysqld]
and
[api]
sections of a
config.ini
file for configuring MySQL
Cluster SQL nodes and API nodes. For detailed descriptions and
other additional information about each of these parameters, see
Section 16.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.
For a discussion of MySQL server options for MySQL Cluster, see Section 16.3.4.2, “MySQL Server Options for MySQL Cluster”; for information about MySQL server system variables relating to MySQL Cluster, see Section 16.3.4.3, “MySQL Cluster System Variables”.
Restart types. Changes in MySQL Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary table as follows:
N
—Node restart: The parameter can be updated using a rolling restart (see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”).S
—System restart: The cluster must be shut down completely, then restarted, to effect a change in this parameter.I
—Initial restart: Data nodes must be restarted using the--initial
option.
For more information about restart types, see Section 16.3.3, “Overview of MySQL Cluster Configuration Parameters”.
Table 16.3. SQL Node/API Node Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
ArbitrationDelay | milliseconds | 4G | N | ||
ArbitrationRank | 0-2 | 2 | N | ||
AutoReconnect | false | false | true | N | |
BatchByteSize | bytes | 32K | 1024 | 1M | N |
BatchSize | records | 64 | 1 | 992 | N |
ConnectionMap | N | ||||
DefaultOperationRedoProblemAction | |||||
ExecuteOnComputer | name | S | |||
HeartbeatThreadPriority | none | ||||
HostName | name or IP address | S | |||
Id | unsigned | 1 | 63 | N | |
MaxScanBatchSize | bytes | 256K | 32K | 16M | N |
NodeId | unsigned | 1 | 63 | N | |
TotalSendBufferMemory | bytes | 256K | 4G | N | |
wan | false | N |
To add new SQL or API nodes to the configuration of a running
MySQL Cluster, it is necessary to perform a rolling restart of
all cluster nodes after adding new [mysqld]
or [api]
sections to the
config.ini
file (or files, if you are
using more than one management server). This must be done
before the new SQL or API nodes can connect to the cluster.
It is not necessary to perform any restart of the cluster if new SQL or API nodes can employ previously unused API slots in the cluster configuration to connect to the cluster.
The summary tables in this section provide information about
parameters used in the [computer]
,
[tcp]
, [shm]
, and
[sci]
sections of a
config.ini
file for configuring MySQL
Cluster management nodes. For detailed descriptions and other
additional information about individual parameters, see
Section 16.3.2.8, “MySQL Cluster TCP/IP Connections”,
Section 16.3.2.10, “MySQL Cluster Shared-Memory Connections”, or
Section 16.3.2.11, “SCI Transport Connections in MySQL Cluster”, as appropriate.
Restart types. Changes in MySQL Cluster configuration parameters do not take effect until the cluster is restarted. The type of restart required to change a given parameter is indicated in the summary tables as follows:
N
—Node restart: The parameter can be updated using a rolling restart (see Section 16.5.4, “Performing a Rolling Restart of a MySQL Cluster”).S
—System restart: The cluster must be shut down completely, then restarted, to effect a change in this parameter.I
—Initial restart: Data nodes must be restarted using the--initial
option.
For more information about restart types, see Section 16.3.3, “Overview of MySQL Cluster Configuration Parameters”.
Table 16.4. COMPUTER Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
HostName | name or IP address | S | |||
Id | string | IN |
Table 16.5. TCP Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
Checksum | false | N | |||
Group | unsigned | 55 | 200 | N | |
NodeId1 | N | ||||
NodeId2 | N | ||||
NodeIdServer | N | ||||
OverloadLimit | bytes | 4G | N | ||
PortNumber | unsigned | 64K | N | ||
Proxy | N | ||||
ReceiveBufferMemory | bytes | 64K | 16K | 4G | N |
SendBufferMemory | unsigned | 256K | 64K | 4G | N |
SendSignalId | false (debug builds: true) | N | |||
TCP_MAXSEG_SIZE | unsigned | 2G | N | ||
TCP_RCV_BUF_SIZE | unsigned | 70080 | 1 | 2G | N |
TCP_SND_BUF_SIZE | unsigned | 71540 | 1 | 2G | N |
TcpBind_INADDR_ANY | false | N |
Table 16.6. SHM Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
Checksum | true | N | |||
Group | unsigned | 35 | 200 | N | |
NodeId1 | N | ||||
NodeId2 | N | ||||
NodeIdServer | N | ||||
OverloadLimit | bytes | 4G | N | ||
PortNumber | unsigned | 64K | N | ||
SendSignalId | false | N | |||
ShmKey | unsigned | 4G | N | ||
ShmSize | bytes | 1M | 64K | 4G | N |
Signum | unsigned | 4G | N |
Table 16.7. SCI Configuration Parameters
Name | Type/Units | Default | Min Value | Max Value | Restart Type |
---|---|---|---|---|---|
Checksum | false | N | |||
Group | unsigned | 15 | 200 | N | |
Host1SciId0 | unsigned | 4G | N | ||
Host1SciId1 | unsigned | 4G | N | ||
Host2SciId0 | unsigned | 4G | N | ||
Host2SciId1 | unsigned | 4G | N | ||
NodeId1 | N | ||||
NodeId2 | N | ||||
NodeIdServer | N | ||||
OverloadLimit | bytes | 4G | N | ||
PortNumber | unsigned | 64K | N | ||
SendLimit | unsigned | 8K | 128 | 32K | N |
SendSignalId | true | N | |||
SharedBufferSize | unsigned | 10M | 64K | 4G | N |
This section provides information about MySQL server options, server and status variables that are specific to MySQL Cluster. For general information on using these, and for other options and variables not specific to MySQL Cluster, see Section 5.1, “The MySQL Server”.
For MySQL Cluster configuration parameters used in the cluster
confiuration file (usually named config.ini
),
see Section 16.3, “MySQL Cluster Configuration”.
The following table provides a list of the command-line options,
server and status variables applicable within
mysqld
when it is running as an SQL node in a
MySQL Cluster. For a table showing all
command-line options, server and status variables available for
use with mysqld, see
Section 5.1.1, “Server Option and Variable Reference”.
Table 16.8. MySQL Server Option and Variable Reference for MySQL Cluster
This section provides descriptions of mysqld server options relating to MySQL Cluster. For information about mysqld options not specific to MySQL Cluster, and for general information about the use of options with mysqld, see Section 5.1.2, “Server Command Options”.
For information about command-line options used with other MySQL
Cluster processes (ndbd,
ndb_mgmd, and ndb_mgm),
see Section 16.4.23, “Options Common to MySQL Cluster Programs”. For
information about command-line options used with
NDB
utility programs (such as
ndb_desc, ndb_size.pl, and
ndb_show_tables), see
Section 16.4, “MySQL Cluster Programs”.
Command-Line Format --ndb-batch-size
Option-File Format ndb-batch-size
Variable Name ndb_batch_size
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 32768
Range 0 .. 31536000
This sets the size in bytes that is used for NDB transaction batches.
--ndb-cluster-connection-pool=
#
Command-Line Format --ndb-cluster-connection-pool
Option-File Format ndb-cluster-connection-pool
Variable Name ndb_cluster_connection_pool
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 1
Range 1 .. 63
By setting this option to a value greater than 1 (the default), a mysqld process can use multiple connections to the cluster, effectively mimicking several SQL nodes. Each connection requires its own
[api]
or[mysqld]
section in the cluster configuration (config.ini
) file, and counts against the maximum number of API connections supported by the cluster.Suppose that you have 2 cluster host computers, each running an SQL node whose mysqld process was started with
--ndb-cluster-connection-pool=4
; this means that the cluster must have 8 API slots available for these connections (instead of 2). All of these connections are set up when the SQL node connects to the cluster, and are allocated to threads in a round-robin fashion.This option is useful only when running mysqld on host machines having multiple CPUs, multiple cores, or both. For best results, the value should be smaller than the total number of cores available on the host machine. Setting it to a value greater than this is likely to degrade performance severely.
ImportantBecause each SQL node using connection pooling occupies multiple API node slots—each slot having its own node ID in the cluster—you must not use a node ID as part of the cluster connectstring when starting any mysqld process that employs connection pooling.
Setting a node ID in the connectstring when using the
--ndb-cluster-connection-pool
option causes node ID allocation errors when the SQL node attempts to connect to the cluster.ЗамечаниеIn some older relases of MySQL Cluster prior to MySQL Cluster NDB 7.2, there was also a separate status variable corresponding to this option; however, the status variable was removed as redundant as of these versions. (Bug #60119)
--ndb-blob-read-batch-bytes=
bytes
Command-Line Format --ndb-blob-read-batch-bytes
Option-File Format ndb-blob-read-batch-bytes
Variable Name ndb_blob_read_batch_bytes
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 65535
Range 0 .. 4294967295
This option can be used to set the size (in bytes) for batching of
BLOB
data reads in MySQL Cluster applications. When this batch size is exceeded by the amount ofBLOB
data to be read within the current transaction, any pendingBLOB
read operations are immediately executed.The maximum value for this option is 4294967295; the default is 65535. Setting it to 0 has the effect of disabling
BLOB
read batching.ЗамечаниеIn NDB API applications, you can control
BLOB
write batching with thesetMaxPendingBlobReadBytes()
andgetMaxPendingBlobReadBytes()
methods.--ndb-blob-write-batch-bytes=
bytes
Command-Line Format --ndb-blob-write-batch-bytes
Option-File Format ndb-blob-write-batch-bytes
Variable Name ndb_blob_write_batch_bytes
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 65535
Range 0 .. 4294967295
This option can be used to set the size (in bytes) for batching of
BLOB
data writes in MySQL Cluster applications. When this batch size is exceeded by the amount ofBLOB
data to be written within the current transaction, any pendingBLOB
write operations are immediately executed.The maximum value for this option is 4294967295; the default is 65535. Setting it to 0 has the effect of disabling
BLOB
write batching.ЗамечаниеIn NDB API applications, you can control
BLOB
write batching with thesetMaxPendingBlobWriteBytes()
andgetMaxPendingBlobWriteBytes()
methods.--ndb-connectstring=
connect_string
Command-Line Format --ndb-connectstring
Option-File Format ndb-connectstring
Permitted Values Type string
When using the
NDBCLUSTER
storage engine, this option specifies the management server that distributes cluster configuration data. See Section 16.3.2.3, “The MySQL Cluster Connectstring”, for syntax.--ndb-deferred-constraints=[TRUE|FALSE]
Command-Line Format --ndb-deferred-constraints
Option-File Format ndb-deferred-constraints
Option Sets Variable Yes, ndb_deferred_constraints
Variable Name ndb-deferred-constraints
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default false
Range false .. true
Controls whether or not constraint checks are deferred, where these are supported.
OFF
is the default.This option was added in MySQL Cluster NDB 7.0.28 and MySQL Cluster NDB 7.1.17.
--ndb-distribution=[KEYHASH|LINHASH]
Command-Line Format --ndb-distribution={KEYHASH|LINHASH}
Option-File Format ndb-distribution
Option Sets Variable Yes, ndb_distribution
Variable Name ndb-distribution
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type enumeration
Default KEYHASH
Valid Values LINHASH
KEYHASH
Controls the default distribution method for
NDB
tables. Can be set to either ofKEYHASH
(key hashing) orLINHASH
(linear hashing).KEYHASH
is the default.Command-Line Format --ndb-mgmd-host=host[:port]
Option-File Format ndb-mgmd-host
Permitted Values Type string
Default localhost:1186
Can be used to set the host and port number of a single management server for the program to connect to. If the program requires node IDs or references to multiple management servers (or both) in its connection information, use the
--ndb-connectstring
option instead.Command-Line Format --ndbcluster
Option-File Format ndbcluster
Option Sets Variable Yes, have_ndbcluster
Disabled by skip-ndbcluster
Permitted Values Type boolean
Default FALSE
The
NDBCLUSTER
storage engine is necessary for using MySQL Cluster. If a mysqld binary includes support for theNDBCLUSTER
storage engine, the engine is disabled by default. Use the--ndbcluster
option to enable it. Use--skip-ndbcluster
to explicitly disable the engine.Command-Line Format --ndb-log-apply-status
Option-File Format ndb-log-apply-status
Option Sets Variable Yes, ndb_log_apply_status
Variable Name ndb_log_apply_status
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
Causes a slave mysqld to log any updates received from its immediate master to the
mysql.ndb_apply_status
table in its own binary log using its own server ID rather than the server ID of the master. In a circular or chain replication setting, this allows such updates to propagate to themysql.ndb_apply_status
tables of any MySQL servers configured as slaves of the current mysqld.In a chain replication setup, using this option allows downstream (slave) clusters to be aware of their positions relative to all of their upstream contributors (masters).
In a circular replication setup, this option causes changes to
ndb_apply_status
tables to complete the entire circuit, eventually propagating back to the originating MySQL Cluster. This also allows a cluster acting as a master to see when its changes (epochs) have been applied to the other clusters in the circle.This option has no effect unless the MySQL server is started with the
--ndbcluster
option.Version Introduced 5.5.15-ndb-7.2.1 Command-Line Format --ndb-log-transaction-id[={0|1}]
Option-File Format ndb-log-transaction-id
Option Sets Variable Yes, ndb_log_transaction_id
Variable Name ndb_log_transaction_id
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
Causes a slave mysqld to write the NDB transaction ID in each row of the binary log. Such logging requires the use of the Version 2 event format for the binary log; thus,
--log-bin-use-v1-row-events
must be set toFALSE
in order to use this option.This option is available beginning with MySQL Cluster NDB 7.2.1 (and is not supported in mainline MySQL Server 5.5). It is required to enable MySQL Cluster Replication conflict detection and resolution using the
NDB$EPOCH_TRANS()
function introduced in the same MySQL Cluster release.The default value is
FALSE
.For more information, see Section 16.6.11, “MySQL Cluster Replication Conflict Resolution”.
Command-Line Format --ndb-nodeid=#
Option-File Format ndb-nodeid
Variable Name Ndb_cluster_node_id
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Range 1 .. 255
Set this MySQL server's node ID in a MySQL Cluster.
The
--ndb-nodeid
option overrides any node ID set with--ndb-connectstring
, regardless of the order in which the two options are used.In addition, if
--ndb-nodeid
is used, then either a matching node ID must be found in a[mysqld]
or[api]
section ofconfig.ini
, or there must be an “open”[mysqld]
or[api]
section in the file (that is, a section without aNodeId
orId
parameter specified). This is also true if the node ID is specified as part of the connectstring.Regardless of how the node ID is determined, its is shown as the value of the global status variable
Ndb_cluster_node_id
in the output ofSHOW STATUS
, and ascluster_node_id
in theconnection
row of the output ofSHOW ENGINE NDBCLUSTER STATUS
.For more information about node IDs for MySQL Cluster SQL nodes, see Section 16.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.
Command-Line Format --ndb-log-empty-epochs
Option-File Format ndb-log-empty-epochs
Variable Name ndb_log_empty_epochs
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default OFF
Causes epochs during which there were no changes to be written to the
ndb_apply_status
andndb_binlog_index
tables, even when--log-slave-updates
is enabled.By default this option is disabled. Disabling
--ndb-log-empty-epochs
causes epoch transactions with no changes not to be written to the binary log, although a row is still written even for an empty epoch inndb_binlog_index
.Because
--ndb-log-empty-epochs=1
causes the size ofndb_binlog_index
table to increase independently of the size of the binary log, users should be prepared to manage the growth of this table, even if they expect the cluster to be idle a large part of the time.Command-Line Format --server-id-bits=#
Option-File Format server-id-bits
Option Sets Variable Yes, server_id_bits
Variable Name server_id_bits
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 32
Range 7 .. 32
This option indicates the number of least significant bits within the 32-bit
server_id
which actually identify the server. Indicating that the server is actually identified by fewer than 32 bits makes it possible for some of the remaining bits to be used for other purposes, such as storing user data generated by applications using the NDB API's Event API within theAnyValue
of anOperationOptions
structure (MySQL Cluster uses theAnyValue
to store the server ID).When extracting the effective server ID from
server_id
for purposes such as detection of replication loops, the server ignores the remaining bits. The--server-id-bits
option is used to mask out any irrelevant bits ofserver_id
in the IO and SQL threads when deciding whether an event should be ignored based on the server ID.This data can be read from the binary log by mysqlbinlog, provided that it is run with its own
--server-id-bits
option set to 32 (the default).The value of
server_id
must be less than 2 ^server_id_bits
; otherwise, mysqld refuses to start.This system variable is supported only by MySQL Cluster. It is not supported in the standard MySQL 5.5 Server.
Command-Line Format --skip-ndbcluster
Option-File Format skip-ndbcluster
Disable the
NDBCLUSTER
storage engine. This is the default for binaries that were built withNDBCLUSTER
storage engine support; the server allocates memory and other resources for this storage engine only if the--ndbcluster
option is given explicitly. See Section 16.3.1, “Quick Test Setup of MySQL Cluster”, for an example.Command-Line Format --ndb-wait-connected=#
Option-File Format ndb-wait-connected
Variable Name ndb-wait-connected
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 0
Range 0 .. 31536000
Permitted Values Type numeric
Default 0
Range 0 .. 31536000
Permitted Values Type numeric
Default 30
Range 0 .. 31536000
Permitted Values Type numeric
Default 30
Range 0 .. 31536000
This option sets the period of time that the MySQL server waits for connections to MySQL Cluster management and data nodes to be established before accepting MySQL client connections. The time is specified in seconds. The default value is
30
.Command-Line Format --ndb-wait-setup=#
Option-File Format ndb-wait-setup
Variable Name ndb-wait-setup
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 15
Range 0 .. 31536000
Permitted Values Type numeric
Default 15
Range 0 .. 31536000
Permitted Values Type numeric
Default 30
Range 0 .. 31536000
Permitted Values Type numeric
Default 30
Range 0 .. 31536000
This variable shows the period of time that the MySQL server waits for the
NDB
storage engine to complete setup before timing out and treatingNDB
as unavailable. The time is specified in seconds. The default value is30
.--ndb_optimization_delay=
milliseconds
Variable Name ndb_optimization_delay
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 10
Range 0 .. 100000
Set the number of milliseconds to wait between sets of rows by
OPTIMIZE TABLE
statements onNDB
tables. The default is 15.
This section provides detailed information about MySQL server
system variables that are specific to MySQL Cluster and the
NDB
storage engine. For system
variables not specific to MySQL Cluster, see
Section 5.1.3, “Server System Variables”. For general
information on using system variables, see
Section 5.1.4, “Using System Variables”.
Variable Name have_ndbcluster
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
YES
if mysqld supportsNDBCLUSTER
tables.DISABLED
if--skip-ndbcluster
is used.This variable is deprecated and is removed in MySQL 5.6. Use
SHOW ENGINES
instead.Command-Line Format --multi_range_count=#
Option-File Format multi_range_count
Option Sets Variable Yes, multi_range_count
Variable Name multi_range_count
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 256
Range 1 .. 4294967295
The maximum number of ranges to send to a table handler at once during range selects. The default value is 256. Sending multiple ranges to a handler at once can improve the performance of certain selects dramatically. This is especially true for the
NDBCLUSTER
table handler, which needs to send the range requests to all nodes. Sending a batch of those requests at once reduces communication costs significantly.This variable is deprecated in MySQL 5.1, and is no longer supported in MySQL 5.5, in which arbitrarily long lists of ranges can be processed.
Command-Line Format --ndb_autoincrement_prefetch_sz
Option-File Format ndb_autoincrement_prefetch_sz
Option Sets Variable Yes, ndb_autoincrement_prefetch_sz
Variable Name ndb_autoincrement_prefetch_sz
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 1
Range 1 .. 256
Determines the probability of gaps in an autoincremented column. Set it to
1
to minimize this. Setting it to a high value for optimization—makes inserts faster, but decreases the likelihood that consecutive autoincrement numbers will be used in a batch of inserts. Default value:32
. Minimum value:1
.This variable affects only the number of
AUTO_INCREMENT
IDs that are fetched between statements; within a given statement, at least 32 IDs are obtained at a time. The default value forndb_autoincrement_prefetch_sz
is1
, to increase the speed of statements inserting single rows.The maximum value for
ndb_autoincrement_prefetch_sz
is 65536.Command-Line Format --ndb_cache_check_time
Option-File Format ndb_cache_check_time
Option Sets Variable Yes, ndb_cache_check_time
Variable Name ndb_cache_check_time
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
The number of milliseconds that elapse between checks of MySQL Cluster SQL nodes by the MySQL query cache. Setting this to 0 (the default and minimum value) means that the query cache checks for validation on every query.
The recommended maximum value for this variable is 1000, which means that the check is performed once per second. A larger value means that the check is performed and possibly invalidated due to updates on different SQL nodes less often. It is generally not desirable to set this to a value greater than 2000.
Command-Line Format --ndb-deferred-constraints
Option-File Format ndb_deferred_constraints
Option Sets Variable Yes, ndb_deferred_constraints
Variable Name ndb_deferred_constraints
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default false
Range false .. true
Controls whether or not constraint checks are deferred, where these are supported.
OFF
is the default.Command-Line Format --ndb-distribution={KEYHASH|LINHASH}
Option-File Format ndb_distribution
Variable Name ndb_distribution={KEYHASH|LINHASH}
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type enumeration
Default KEYHASH
Valid Values LINHASH
KEYHASH
Controls the default distribution method for
NDB
tables. Can be set to either ofKEYHASH
(key hashing) orLINHASH
(linear hashing).KEYHASH
is the default.Command-Line Format ndb_extra_logging=#
Option-File Format ndb_extra_logging
Variable Name ndb_extra_logging
Variable Scope Global Dynamic Variable Yes Permitted Values Type numeric
Default 0
This variable can be used to enable recording in the MySQL error log of information specific to the
NDB
storage engine. It is normally of interest only when debuggingNDB
storage engine code.The default value is 0, which means that the only
NDB
-specific information written to the MySQL error log relates to transaction handling. If the value is greater than 0 but less than 10,NDB
table schema and connection events are also logged, as well as whether or not conflict resolution is in use, and otherNDB
errors and information. If the value is set to 10 or more, information aboutNDB
internals, such as the progress of data distribution among cluster nodes, is also written to the MySQL error log.Command-Line Format --ndb-force-send
Option-File Format ndb_force_send
Option Sets Variable Yes, ndb_force_send
Variable Name ndb_force_send
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default TRUE
Forces sending of buffers to
NDB
immediately, without waiting for other threads. Defaults toON
.Command-Line Format --ndb_index_stat_cache_entries
Option-File Format ndb_index_stat_cache_entries
Permitted Values Type numeric
Default 32
Range 0 .. 4294967295
Sets the granularity of the statistics by determining the number of starting and ending keys to store in the statistics memory cache. Zero means no caching takes place; in this case, the data nodes are always queried directly. Default value:
32
.ЗамечаниеIf
ndb_index_stat_enable
isOFF
, then setting this variable has no effect.Command-Line Format --ndb_index_stat_enable
Option-File Format ndb_index_stat_enable
Permitted Values Type boolean
Default ON
Use
NDB
index statistics in query optimization. Defaults toON
.Command-Line Format --ndb_index_stat_update_freq
Option-File Format ndb_index_stat_update_freq
Permitted Values Type numeric
Default 20
Range 0 .. 4294967295
How often to query data nodes instead of the statistics cache. For example, a value of
20
(the default) means to direct every 20th query to the data nodes.ЗамечаниеIf
ndb_index_stat_cache_entries
is0
, then setting this variable has no effect; in this case, every query is sent directly to the data nodes.Variable Name ndb_join_pushdown
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default TRUE
Added in MySQL Cluster NDB 7.2.0, this variable controls whether joins on
NDB
tables are pushed down to the NDB kernel (data nodes). Previously, a join was handled using multiple accesses ofNDB
by the SQL node; however, whenndb_join_pushdown
is enabled, a pushable join is sent in its entirety to the data nodes, where it can be distributed among the data nodes and executed in parallel on multiple copies of the data, with a single, merged result being returned to mysqld. This can reduce greatly the number of round trips between an SQL node and the data nodes required to handle such a join.By default,
ndb_join_pushdown
is enabled.In order for a join to be pushable, it must meet the following conditions:
Only columns can be compared, and all columns to be joined must use exactly the same data type.
This means that expressions such as
t1.a = t2.a +
cannot be pushed down, and that (for example) a join on anconstant
INT
column and aBIGINT
column also cannot be pushed down.Explicit locking is not supported; however, the
NDB
storage engine's characteristic implicit row-based locking is enforced.This means that a join using
FOR UPDATE
cannot be pushed down.In order for a join to be pushed down, child tables in the join must be accessed using one of the
ref
,eq_ref
, orconst
access methods, or some combination of these methods.Outer joined child tables can only be pushed using
eq_ref
.If the root of the pushed join is an
eq_ref
orconst
, only child tables joined byeq_ref
can be appended. (A table joined byref
is likely to become the root of another pushed join.)If the query optimizer decides on
Using join cache
for a candidate child table, that table cannot be pushed as a child. However, it may be the root of another set of pushed tables.Joins referencing tables explicitly partitioned by
[LINEAR] HASH
,LIST
, orRANGE
currently cannot be pushed down.
You can see whether a given join can be pushed down by checking it with
EXPLAIN
; when the join can be pushed down, you can see references to thepushed join
in theExtra
column of the output, as shown in this example:mysql>
EXPLAIN
->SELECT e.first_name, e.last_name, t.title, d.dept_name
->FROM employees e
->JOIN dept_emp de ON e.emp_no=de.emp_no
->JOIN departments d ON d.dept_no=de.dept_no
->JOIN titles t ON e.emp_no=t.emp_no\G
*************************** 1. row *************************** id: 1 select_type: SIMPLE table: d type: ALL possible_keys: PRIMARY key: NULL key_len: NULL ref: NULL rows: 9 Extra: Parent of 4 pushed join@1 *************************** 2. row *************************** id: 1 select_type: SIMPLE table: de type: ref possible_keys: PRIMARY,emp_no,dept_no key: dept_no key_len: 4 ref: employees.d.dept_no rows: 5305 Extra: Child of 'd' in pushed join@1 *************************** 3. row *************************** id: 1 select_type: SIMPLE table: e type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: employees.de.emp_no rows: 1 Extra: Child of 'de' in pushed join@1 *************************** 4. row *************************** id: 1 select_type: SIMPLE table: t type: ref possible_keys: PRIMARY,emp_no key: emp_no key_len: 4 ref: employees.de.emp_no rows: 19 Extra: Child of 'e' in pushed join@1 4 rows in set (0.00 sec)ЗамечаниеIf inner joined child tables are joined by
ref
, and the result is ordered or grouped by a sorted index, this index cannot provide sorted rows, which forces writing to a sorted tempfile.Two additional sources of information about pushed join performance are available:
The status variables
Ndb_pushed_queries_defined
,Ndb_pushed_queries_dropped
,Ndb_pushed_queries_executed
, andNdb_pushed_reads
(all introduced in MySQL Cluster NDB 7.2.0).The counters in the
ndbinfo.counters
table that belong to theDBSPJ
kernel block. (These counters and theDBSPJ
block were also introduced in MySQL Cluster NDB 7.2.0). See Section 16.5.9.3, “Thendbinfo counters
Table”, for information about these counters. See also TheDBSPJ
Block, in the MySQL Cluster API Developer Guide.
Command-Line Format --ndb-log-apply-status
Option-File Format ndb-log-apply-status
Option Sets Variable Yes, ndb_log_apply_status
Variable Name ndb_log_apply_status
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
A read-only variable which shows whether the server was started with the
--ndb-log-apply-status
option.Command-Line Format --ndb-log-bin={1|0}
Option Sets Variable Yes, ndb_log_bin
Variable Name ndb_log_bin
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default ON
Causes updates to
NDB
tables to be written to the binary log. Setting this variable has no effect if binary logging is not already enabled for the server usinglog_bin
.ndb_log_bin
defaults to 1 (ON); normally, there is never any need to change this value in a production environment.Command-Line Format --ndb-log-binlog-index={1|0}
Option Sets Variable Yes, ndb_log_binlog_index
Variable Name ndb_log_binlog_index
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default ON
Causes a mapping of epochs to positions in the binary log to be inserted into the
ndb_binlog_index
table. Setting this variable has no effect if binary logging is not already enabled for the server usinglog_bin
. (In addition,ndb_log_bin
must not be disabled.)ndb_log_binlog_index
defaults to1
(ON
); normally, there is never any need to change this value in a production environment.Version Introduced 5.5.15-ndb-7.2.1 Variable Name ndb_log_transaction_id
Variable Scope Global Dynamic Variable No Permitted Values Type boolean
Default OFF
This read-only, Boolean system variable shows whether a slave mysqld writes NDB transaction IDs in the binary log (required to use “active-active” MySQL Cluster Replication with
NDB$EPOCH_TRANS()
conflict detection). To change the setting, use the--ndb-log-transaction-id
option.ndb_log_transaction_id
is available in MySQL Cluster NDB 7.2.1 and later. It is not supported in mainline MySQL Server 5.5.For more information, see Section 16.6.11, “MySQL Cluster Replication Conflict Resolution”.
Command-Line Format --ndb-optimized-node-selection
4.1.9-5.1.22-ndb-6.33 --ndb-optimized-node-selection=#
Option-File Format ndb_optimized_node_selection
Permitted Values Type boolean
Default ON
Permitted Values Type numeric
Default 3
Range 0 .. 3
There are two forms of optimized node selection, described here:
The SQL node uses promixity to determine the transaction coordinator; that is, the “closest” data node to the SQL node is chosen as the transaction coordinator. For this purpose, a data node having a shared memory connection with the SQL node is considered to be “closest” to the SQL node; the next closest (in order of decreasing proximity) are: TCP connection to
localhost
; SCI connection; TCP connection from a host other thanlocalhost
.The SQL thread uses distribution awareness to select the data node. That is, the data node housing the cluster partition accessed by the first statement of a given transaction is used as the transaction coordinator for the entire transaction. (This is effective only if the first statement of the transaction accesses no more than one cluster partition.)
This option takes one of the integer values
0
,1
,2
, or3
.3
is the default. These values affect node selection as follows:0
: Node selection is not optimized. Each data node is employed as the transaction coordinator 8 times before the SQL thread proceeds to the next data node.1
: Proximity to the SQL node is used to determine the transaction coordinator.2
: Distribution awareness is used to select the transaction coordinator. However, if the first statement of the transaction accesses more than one cluster partition, the SQL node reverts to the round-robin behavior seen when this option is set to0
.3
: If distribution awareness can be employed to determine the transaction coordinator, then it is used; otherwise proximity is used to select the transaction coordinator. (This is the default behavior.)
ndb_report_thresh_binlog_epoch_slip
Command-Line Format --ndb_report_thresh_binlog_epoch_slip
Option-File Format ndb_report_thresh_binlog_epoch_slip
Permitted Values Type numeric
Default 3
Range 0 .. 256
This is a threshold on the number of epochs to be behind before reporting binary log status. For example, a value of
3
(the default) means that if the difference between which epoch has been received from the storage nodes and which epoch has been applied to the binary log is 3 or more, a status message will be sent to the cluster log.ndb_report_thresh_binlog_mem_usage
Command-Line Format --ndb_report_thresh_binlog_mem_usage
Option-File Format ndb_report_thresh_binlog_mem_usage
Permitted Values Type numeric
Default 10
Range 0 .. 10
This is a threshold on the percentage of free memory remaining before reporting binary log status. For example, a value of
10
(the default) means that if the amount of available memory for receiving binary log data from the data nodes falls below 10%, a status message will be sent to the cluster log.Command-Line Format --slave-allow-batching
Option-File Format slave_allow_batching
Option Sets Variable Yes, slave_allow_batching
Variable Name slave_allow_batching
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default off
Whether or not batched updates are enabled on MySQL Cluster replication slaves.
This variable is available beginning with MySQL Cluster NDB 6.2.3. Currently, it is available for mysqld only as supplied with MySQL Cluster or built from the MySQL Cluster sources. For more information, see Section 16.6.6, “Starting MySQL Cluster Replication (Single Replication Channel)”.
Variable Name ndb_table_no_logging
Variable Scope Session Dynamic Variable Yes Permitted Values Type boolean
Default FALSE
When this variable is set to
ON
or1
, it causesNDB
tables not to be checkpointed to disk. More specifically, this setting applies to tables which are created or altered usingENGINE NDB
whenndb_table_no_logging
is enabled, and continues to apply for the lifetime of the table, even ifndb_table_no_logging
is later changed. Suppose thatA
,B
,C
, andD
are tables that we create (and perhaps also alter), and that we also change the setting forndb_table_no_logging
as shown here:SET @@ndb_table_no_logging = 1; CREATE TABLE A ... ENGINE NDB; CREATE TABLE B ... ENGINE MYISAM; CREATE TABLE C ... ENGINE MYISAM; ALTER TABLE B ENGINE NDB; SET @@ndb_table_no_logging = 0; CREATE TABLE D ... ENGINE NDB; ALTER TABLE C ENGINE NDB; SET @@ndb_table_no_logging = 1;
After the previous sequence of events, tables
A
andB
are not checkpointed;A
was created withENGINE NDB
and B was altered to useNDB
, both whilendb_table_no_logging
was enabled. However, tablesC
andD
are logged;C
was altered to useNDB
andD
was created usingENGINE NDB
, both whilendb_table_no_logging
was disabled. Settingndb_table_no_logging
back to1
orON
does not cause tableC
orD
to be checkpointed.Замечаниеndb_table_no_logging
has no effect on the creation ofNDB
table schema files; to suppress these, usendb_table_temporary
instead.Variable Name ndb_table_temporary
Variable Scope Session Dynamic Variable Yes Permitted Values Type boolean
Default FALSE
When set to
ON
or1
, this variable causesNDB
tables not to be written to disk: This means that no table schema files are created, and that the tables are not logged.ЗамечаниеSetting this variable currently has no effect in MySQL Cluster NDB 7.0 and later. This is a known issue; see BUG#34036.
Variable Name ndb_use_copying_alter_table
Variable Scope Global, Session Dynamic Variable No Forces
NDB
to use copying of tables in the event of problems with onlineALTER TABLE
operations. The default value isOFF
.Variable Name ndb_use_exact_count
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default ON
Forces
NDB
to use a count of records duringSELECT COUNT(*)
query planning to speed up this type of query. The default value isON
. For faster queries overall, disable this feature by setting the value ofndb_use_exact_count
toOFF
.Command-Line Format --ndb_use_transactions
Option-File Format ndb_use_transactions
Variable Name ndb_use_transactions
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default ON
You can disable
NDB
transaction support by setting this variable's values toOFF
(not recommended). The default isON
.Variable Name transaction_allow_batching
Variable Scope Session Dynamic Variable Yes Permitted Values Type boolean
Default FALSE
When set to
1
orON
, this variable enables batching of statements within the same transaction. To use this variable,autocommit
must first be disabled by setting it to0
orOFF
; otherwise, settingtransaction_allow_batching
has no effect.It is safe to use this variable with transactions that performs writes only, as having it enabled can lead to reads from the “before” image. You should ensure that any pending transactions are committed (using an explicit
COMMIT
if desired) before issuing aSELECT
.Importanttransaction_allow_batching
should not be used whenever there is the possibility that the effects of a given statement depend on the outcome of a previous statement within the same transaction.This variable is currently supported for MySQL Cluster only.
The system variables in the following list all relate to the
ndbinfo
information database.
Variable Name ndbinfo_database
Variable Scope Global Dynamic Variable No Permitted Values Type string
Default ndbinfo
Shows the name used for the
NDB
information database; the default isndbinfo
. This is a read-only variable whose value is determined at compile time; you can set it by starting the server using--ndbinfo-database=
, which sets the value shown for this variable but does not actually change the name used for the NDB information database.name
Command-Line Format --ndbinfo-max-bytes=#
Option Sets Variable Yes, ndbinfo_max_bytes
Variable Name ndbinfo_max_bytes
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 0
Used in testing and debugging only.
Command-Line Format --ndbinfo-max-rows=#
Option Sets Variable Yes, ndbinfo_max_rows
Variable Name ndbinfo_max_rows
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type numeric
Default 10
Used in testing and debugging only.
Command-Line Format --ndbinfo-show-hidden={0|1}
Option Sets Variable Yes, ndbinfo_show_hidden
Variable Name ndbinfo_show_hidden
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type boolean
Default no
Whether or not the
ndbinfo
database's underlying internal tables are shown in themysql
client. The default isOFF
.Command-Line Format --ndbinfo-table-prefix=name
Option Sets Variable Yes, ndbinfo_table_prefix
Variable Name ndbinfo_table_prefix
Variable Scope Global, Session Dynamic Variable Yes Permitted Values Type string
Default ndb$
The prefix used in naming the ndbinfo database's base tables (normally hidden, unless exposed by setting
ndbinfo_show_hidden
). This is a read-only variable whose default value is “ndb$
”. You can start the server with the--ndbinfo-table-prefix
option, but this merely sets the variable and does not change the actual prefix used to name the hidden base tables; the prefix itself is determined at compile time.Variable Name ndbinfo_version
Variable Scope Global Dynamic Variable No Permitted Values Type string
Default Shows the version of the
ndbinfo
engine in use; read-only.Command-Line Format --ndb-log-empty-epochs
Option-File Format ndb_log_empty_epochs
Variable Name ndb_log_empty_epochs
Variable Scope Global Dynamic Variable Yes Permitted Values Type boolean
Default OFF
When this variable is set to 0, epoch transactions with no changes are not written to the binary log, although a row is still written even for an empty epoch in
ndb_binlog_index
.Command-Line Format --server-id-bits=#
Option-File Format server-id-bits
Option Sets Variable Yes, server_id_bits
Variable Name server_id_bits
Variable Scope Global Dynamic Variable No Permitted Values Type numeric
Default 32
Range 7 .. 32
The effective value of
server_id
if the server was started with the--server-id-bits
option set to a nondefault value.If the value of
server_id
greater than or equal to 2 to the power ofserver_id_bits
, mysqld refuses to start.This system variable is supported only by MySQL Cluster.
server_id_bits
is not supported by the standard MySQL Server.
This section provides detailed information about MySQL server
status variables that relate to MySQL Cluster and the
NDB
storage engine. For status
variables not specific to MySQL Cluster, and for general
information on using status variables, see
Section 5.1.5, “Server Status Variables”.
The MySQL server can ask the
NDBCLUSTER
storage engine if it knows about a table with a given name. This is called discovery.Handler_discover
indicates the number of times that tables have been discovered using this mechanism.Ndb_api_wait_exec_complete_count_session
The number of times a thread has been blocked in this client session while waiting for execution of an operation to complete. This includes all
execute()
calls as well as implicit implicit executes for blob and auto-increment operations not visible to clients.Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it relates to the current session only, and is not affected by any other clients of this mysqld.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_exec_complete_count_slave
The number of times a thread has been blocked by this slave while waiting for execution of an operation to complete. This includes all
execute()
calls as well as implicit implicit executes for blob and auto-increment operations not visible to clients.Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it is effectively global in scope. If this MySQL server does not act as a replication slave, or does not use NDB tables, this value is always 0.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_exec_complete_count
The number of times a thread has been blocked by this MySQL Server (SQL node) while waiting for execution of an operation to complete. This includes all
execute()
calls as well as implicit implicit executes for blob and auto-increment operations not visible to clients.Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it is effectively global in scope.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_scan_result_count_session
The number of times a thread has been blocked in this client session while waiting for a scan-based signal, such as when waiting for more results from a scan, or when waiting for a scan to close.
Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it relates to the current session only, and is not affected by any other clients of this mysqld.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_scan_result_count_slave
The number of times a thread has been blocked by this slave while waiting for a scan-based signal, such as when waiting for more results from a scan, or when waiting for a scan to close.
Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it is effectively global in scope. If this MySQL server does not act as a replication slave, or does not use NDB tables, this value is always 0.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_scan_result_count
The number of times a thread has been blocked by this MySQL Server (SQL node) while waiting for a scan-based signal, such as when waiting for more results from a scan, or when waiting for a scan to close.
Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it is effectively global in scope.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_meta_request_count_session
The number of times a thread has been blocked in this client session waiting for a metadata-based signal, such as is expected for DDL requests, new epochs, and seizure of transaction records.
Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it relates to the current session only, and is not affected by any other clients of this mysqld.For more information, see Section 16.5.14, “NDB API Statistics Counters and Variables”.
Ndb_api_wait_meta_request_count_slave
The number of times a thread has been blocked by this slave waiting for a metadata-based signal, such as is expected for DDL requests, new epochs, and seizure of transaction records.
Although this variable can be read using either
SHOW GLOBAL STATUS
orSHOW SESSION STATUS
, it is effectively global in scope. If this MyS