-
- Weitere Informationen zu diesem Buch:
Inhaltsverzeichnis | Index | Probekapitel | Kolophon | Rezensionen |
- Weitere Informationen zu diesem Buch:
Optimization, Backups, Replication, Load Balancing & More
First Edition April 2004
ISBN 978-0-596-00306-7
Weitere Informationen zu diesem Buch
Inhaltsverzeichnis |
Index |
Probekapitel |
Kolophon |
Rezensionen |
Index
[ Symbols ], [ A ], [ B ], [ C ], [ D ], [ E ], [ F ], [ G ], [ H ], [ I ], [ J ], [ K ], [ L ], [ M ], [ N ], [ O ], [ P ], [ Q ], [ R ], [ S ], [ T ], [ U ], [ V ], [ W ], [ X ]
A[ Top ]
access control, 204
(see also privileges)
ACID (Atomicity, Consistency, Isolation, and Durability), 22
ACID transactions, 22-29
advantages and disadvantages, 23
deadlocks, 26
isolation levels, 23-26
in MySQL, 27
transaction logging, 27
ALTER TABLE statement, 33
ANALYZE TABLE command, 78, 84
ANSI SQL isolation levels, 26
application-level encryption, 233-234
auditing, 188
authentication, 204
authorization, 204
AUTOCOMMIT mode, 27
automatic host blocking, 230
average employee account privileges, 217
B[ Top ]
backups, 187-202
custom scripts, writing, 199-202
data compression and, 192
filesystem snapshots, 199
offline backups, 198
online versus offline, 191
planning, 189-193
dumps versus raw backups, 189
raw backups, 190
reasons for performing, 187-189
auditing, 188
disaster recovery, 188
testing, 189
recovery (see recovery)
replication and, 131, 193
retention of old backups, 192
storage demands and media, 192
table types and consistency, 191
tools, 193-197
InnoDB Hot Backup, 197
mysqldump, 193-195
mysqlhotcopy, 195
mysqlsnapshot, 196
balanced tree (B-tree) indexes, 68
BDB storage engine (see Berkeley DB storage engine)
BDB tables (see Berkeley DB storage engine)
BEGIN statement, 22
bench_example script, 58
benchmarking, 45-60
importance, 46
performance testing versus stress testing, 46
questions answered by, 46
strategies, 46-48
testing methodology, 47
client numbers, 48
client/server separation, 48
isolating the variable, 47
iterative testing, 47
real data, 48
repetition, 47
tools, 49-60
MyBench (see MyBench)
MySQL Benchmark Suite (see MySQL Benchmark Suite)
MySQL super-smack (see MySQL super-smack)
scripting solutions (see MyBench)
Windows and, 49
Berkeley DB (BDB) storage engine, 44
indexes, 73
refreshing statistics, 78
primary keys and, 67
binary installations, MySQL.com binaries versus distribution binaries, 3
binary logs, 130
resetting, 139
B-tree indexes, 68
buffers, 105
bulletin boards, use of storage engines for, 32
C[ Top ]
caching, 105
CPU cache, 107
load-balancing and, 172
MySQL 4.0, cache checking limitations, 80
query cache, 80-81
server tuning and, 123
write caching, 112
RAID and, 112
catalogs, use of storage engines for, 31
CD-ROM applications, use of storage engines for, 33
CHANGE MASTER TO ... command, 151
check_repl script, 154
C-JDBC (Clustered JDBC), 171
clumpy data, 64
cluster partitioning, 179-183
data-based partitioning, 181
filtering, 182
multicluster partitioning, 182
role-based partitioning, 179
clustered indexes, 65, 66
potential for degraded performance, 66
CNAME, 131
code example permissions, xiii
columns_priv table, 207, 214
concurrency (see locking and concurrency)
configuration files, 4-11
client and server section names, 6
command-line arguments, 4
comments, 6
file formats, 6
file locations, 4
key/value pairs, 6
machine size and, 7
reconfiguration, 10
revision control, 10
sample configuration files, 7-10
my-medium.cnf, 7-10
server variables, setting, 10
set-variable statements, 6
connection pooling and load-balancing, 173
CPU speed versus main memory speed, 105
CPUs, solving bottlenecks with, 124
CREATE and SELECT table conversion method, 34
creation id, 21
D[ Top ]
data
clumpy data, 64
distribution via replication, 130
database administrator account privileges, 216
database dumps, 189
database version, 21
db table, 207, 212
DBIx::DBCluster module, 171
deadlocks, 26
delayed key writes, 72
DELAY_KEY_WRITE option, MyISAM storage engine, 39
deletion id, 21
dictionaries, 55
dirty reads, 24
disks
IDE versus SCSI disks, 112
seek time, 104
server performance, impact on, 104
(see also RAID)
DMZs, 225
drop and reimport table conversion method, 34
dual master replication configuration, 146
dumps, 189
disadvantages, 190
duplex mismatch, 106
E[ Top ]
EMIC Networks high availability technology, 186
ENCRYPT( ), 231
exclusive locks, 18
EXPLAIN command, 82-91
output, information in, 83
rows output, 85
ext2 and ext3 filesystems, 115
F[ Top ]
file privilege, 206
filesystems, 113-117
choosing, 115-117
doing without, 116
encryption of, 232
journaling, 114
firewalls, 225
fix_repl script, 155
FLUSH, 199
FreeBSD
filesystems, 116
MySQL distribution, 3
soft updates, 114
threading, 118
UFS and UFS2 filesystems, 116
.frm files, 35
full-text indexes, 74
versus LIKE queries, 95
G[ Top ]
\G escape, 12
gen-data command, 55
global buffers, 105
global privileges, 206
GLOBAL syntax, 10
GRANT command, 215-219
grant privilege, 206
grant tables, 204, 207
fields used for matching records in, 208
H[ Top ]
hard drives (see disks)
hardware, buying, 107
hash indexes, 69
health checks, 175-177
Heap tables, 44
indexes and, 73
primary keys and, 67
high availability, 182-186
commercial solutions, 186
EMIC Networks, 186
Veritas, 186
dual-master replication and load-balancing, 183
and failover through replication, 131
role-based addressing, 132
keepalived software, 185
shared storage with standby, 184
host table, 207, 211
hostname-bin, 136
Hot Backup Tool (ibbackup), 197
hot-swappable RAID disks, 107
HTTP load-balancing, compared to MySQL, 171-174
I[ Top ]
ibbackup (Hot Backup Tool), 197
IDE versus SCSI disks, 112
implicit commits, 27
index privilege, 206
indexes, 61-78
B-tree indexes, 68
clustered indexes, 65
potential for degraded performance, 66
constraints, used as, 65
database indexes, 62
full-text indexes, 74
hash indexes, 69
index order, 64
index structures, 68-72
index-only queries, 86
index-organized tables, 66
keys and, 63
limitations, 75-76
corruption, 76
poor statistics, 76
regular expressions, 75
too many matching rows, 76
wildcard matches, 75
maintenance, 76-78
obtaining information, 77
refreshing statistics, 77
multicolumn indexes, 63
NULLs, 67
partial indexes, 63
queries against the wrong index, 120-122
query optimization and, 92
data, insufficient diversity in, 92
full-text indexes versus LIKE, 95
index-based ordering, 93
index-related hints, 97
R-tree indexes, 70-72
secondary indexes, 65
table types and, 72-76
Berkeley DB (BDB) storage engine, 73
Heap tables, 73
InnoDB storage engine, 74
MyISAM storage engine, 72
tradeoffs, costs versus advantages, 63
unique indexes, 65
versus primary keys, 67
usage, verifying (see EXPLAIN command)
in-memory (Heap) tables (see Heap tables)
Innobase, 9
Innobase Oy, 42
InnoDB Hot Backup tool, 197
InnoDB storage engine, 9, 42-44
buffer pool, 105
clustered indexes, 66
indexes
refreshing statistics, 78
indexes and, 74
locking and concurrency, 43
primary keys and, 67
referential integrity, 43
SHOW INNODB STATUS command, 14
storage, 43
InnoDB tables (see InnoDB storage engine)
innodb_buffer_pool_size, 105
installing MySQL, 2-4
iostat, 124
ISAM (Index Sequential Access Method) table format, 37
isolation levels, 23-26
ANSI SQL, 26
setting, 28
J[ Top ]
JFS filesystem, 116
joins, 91
journaling filesystems, 114
K[ Top ]
keepalived, 185
kernel bottlenecks, solving, 127
key_buffer_size, 105
keys
indexes and, 63
key parts, 64
primary keys, 67
L[ Top ]
LIKE queries versus full-text indexes, 95
Linux
2.4 kernel swapping, 117
distributions of MySQL included in, 3
filesystems, 115
nonjournaling filesystems, 114
Linux Virtual Server project, 131
Linux volume manager (LVM), snapshot capabilities, 199
LOAD DATA INFILE command, 162
security concerns, 206
LOAD TABLE FROM MASTER command, 139
load-balancing, 123, 169-183
availability, 170
caching systems, 172
configuration, 174-179
cluster partitioning, 179-183
connection limits, 177
health checks, 175-177
next-connection algorithms, 177-179
efficiency, 170
load-balancing products, 131
multi-master load-balancing, 174
MySQL and HTTP, compared, 171-174
connection pooling, 173
partitioning, 172
requests, 171
purpose, 169
scalability, 170
software solutions, 171
transparency, 171
using replication, 130
load-balancing products, 131
localhost and MySQL, 224
localhost-only connections, security, 223
LOCK TABLES command, 29
locking and concurrency, 18-22
InnoDB storage engine, 43
lock granularity, 19
lock types, 18
MVCC, 20-22
MyISAM storage engine, 38
page locks, 20
row locks, 20
logging
log corruption, 161
log index files, 142
logfiles, 142
out of sync binary and transaction logs, 163
privileges, setting, 217
storage engines, use for, 31
transaction logging, 27
logins, 203
LVM (Linux volume manager), snapshot capabilities, 199
M[ Top ]
max_connections, 177
McKusick, Kirk, 114
MD5( ), 231
memory
bottlenecks, solving, 125
impact on server performance, 105
Merge tables, 31, 40-42
mirroring, 110
missing temporary tables, problems caused by, 163
multicolumn indexes, 63
multi-master load-balancing, 174
multi-master replication configuration, 146
multiple servers, configuration files for, 5
MVCC (Multi-Version Concurrency Control), 20-22
MyBench, 57-60
bench_example script, 58
fork_and_work( ) subroutine, 59
(see also benchmarking)
.MYD file types, 37
.MYI file types, 37
MyISAM storage engine, 37-42
automatic repair, 38
compressed tables, 39
delayed key writes, 39, 72
full-text indexes, 74
indexes
refreshing statistics, 77
indexes and, 72
indexes in, 65
indexing, 39
key buffer, 105
locking and concurrency, 38
Merge tables, 31, 40-42
packed keys, 72
prefix compression, 72
RAID tables, 39
read queries and, 20
REPAIR TABLE command, 73
row counting speed, 30
storage, 37
table size, 37
MyISAM tables (see MyISAM storage engine)
myisamchk command-line tool, 78
my-medium.cnf file, 7-10
mysnap.pl, 200
output, 202
MySQL
ANSI standard isolation levels, support for, 28
architecture, 16
distributions, compared, 3
installations, binary versus source, 2-4
load-balancing compared to HTTP, 171-174
online documentation, xii
performance, monitoring, 11-15
platforms, xii
support, binary distributions, 3
versions, xii
MySQL 4.0, cache checking limitations, 80
MySQL Benchmark Suite, 49
insert test, 50
logging, 50
multi-CPU machines and, 50
testing non-MySQL servers, 49
(see also benchmarking)
mysql command interpreter, \G escape, 12
MySQL super-smack, 51-57
configuring, 53-57
delim option, 56
dictionaries, 55
example benchmarks, 52
gen-data command, 55
installing, 51
obtaining, 51
smack file, 54
test data, preparing, 53
(see also benchmarking)
mysqladmin command, using for offline backups, 198
mysqlbinlog utility, 137, 153
mysqldiff script, 157
mysqld_safe command line, 11
mysqldump, 34, 189, 193-195
mysqldumpslow, 96
mysqlhotcopy, 190, 195
mysql_pconnect( ), 173
mysql.server startup script, 11
mysqlsnapshot, 190, 196
mytop, 248-256
command-line arguments, 253
common tasks, 255
configuration, 253
display modes, 248-251
command view, 250
status view, 250
thread view, 249-252
installation, 253
long-running queries, management using, 255
query statistics, collection with, 255
required Perl modules, 252
supported operating systems, 251
web site, 248
N[ Top ]
Native POSIX Thread Library (NPTL), 118
NDB storage engine and cluster, 183
Network Appliance filers, 199
Network Attached Storage (NAS) and MySQL, 106
network load-balancing products, 131
Network Operation Center's (NOC) user account, 218
networks
capacity utilization, 106
impact on server performance, 106-109
network ports, 108
security and SSH tunneling, 229
next-connection algorithms, 177-179
next-key locking, 25
non-repeatable reads, 25
nonreplicated dependencies, problems caused by, 162
nonunique server IDs, problems caused by, 161
NPTL (Native POSIX Thread Library), 118
NULLs, 67
primary keys and, 68
O[ Top ]
O(n) problem, 62
Open Source SQL Relay package, 171
OpenGIS specifications, 70
operating systems, 113-118
filesystems, 113-117
security, 222
swap, 117
threading, 118
operations and monitoring personnel, privileges, 218
optimization, 1
(see also query optimizer)
OPTIMIZE TABLE command, 77
order processing, use of storage engines for, 31
P[ Top ]
Pachev, Sasha, 51
packed keys, 72
page locks, 20
parity blocks, 110
partial indexes, 63
partial log records, problems caused by, 161
partitioning, 172
PASSWORD( ), 231
passwords
database-specific passwords, 204
hashing, security advantages of, 231
security concerns, 210
performance, monitoring, 11-15
performance testing versus stress testing, 46
Perl scripts, downloading, xii
phantom rows, 25
phpMyAdmin, 257-264
export and download of data, using for, 262
simple SQL commands, 260
user maintenance using, 259
web site, 257
prefix compression, 72
primary keys versus unique indexes, 67
privileges, 205-207
average employee accounts, 217
database administrator accounts, 216
global privileges, 206
logging applications, 217
operations and monitoring personnel, 218
privilege checks, 207
system administrator account, 215
process privilege, 206
processes, displaying, 12
PURGE MASTER LOGS TO ... command, 151
purge_binary_logs script, 156
pyramid configuration, replication, 147
Q[ Top ]
queries
analysis, 81
execution, 91
EXPLAIN output (see EXPLAIN command)
hashing of, 80
index-only queries, 86
optimization, 81
parsing, 81
preventing caching, 80
processing, 79-92
using the wrong index, 120-122
whitespace in, 80
query barrels, 51
query cache, 80-81
query_cache_type, 80
query optimization versus server tuning, 103
query optimizer, 81, 92-95
data, insufficient diversity in, 92
full-text versus LIKE, 95
impossible queries, 94
index-based ordering, 93
query performance, 79-102
influencing with hints, 96-98
index usage, 97
join order, 96
query cache, 97
result sizes, 97
slow queries, identifying, 95
tricks for improving, 98-102
two queries instead of one, 98-100
unions versus ORs, 100-102
R[ Top ]
RAID (Redundant Array of Inexpensive Disks), 109-113
hardware versus software RAID, 112
hot-swappable disks, 107
IDE versus SCSI disks, 112
mirroring, 110
multiple arrays, using, 111
on slaves, 113
parity blocks, 110
RAID 0, 109
RAID 1, 110
RAID 10 (RAID 1+0), 110
RAID 5, 110
RAID tables (MyISAM storage engine), 39
striping, 109
summary of features, 110
write caching, 112
raw backups, 190
read committed isolation level, 24
read lock, 18
read uncommitted isolation level, 24
read_heartbeat script, 159
read/write locks, 18
reconfiguration, 10
recovery
dumps, restoring from, 189
mysqldump, 194
mysqlhotcopy, 196
mysqlsnapshot, 197
offline backups, 198
references privilege, 206
regular expressions and indexes, 75
Reiser, Hans, 116
ReiserFS filesystem, 116
reload privilege, 206
REPAIR TABLE command, 73
repeatable read isolation level, 25
replication, 1, 129-168
administration and maintenance, 148-160
log rotation, 151
master status, 149
masters, changing, 151-153
monitoring, 149
replication heartbeat, 150
slave status, 150
architectures, 144-148
backups and, 193
common problems, 160-164
binary and transaction logs out of sync, 163
bulk-loading data, 162
hostname changes, caused by, 164
log corruption, 161
missing temporary tables, 163
nonreplicated dependencies, 162
nonunique server IDs, 161
slave data changes, 160
configuring, 134-139
account creation, 135
configuration file entries, 136
custom architectures, 148
dual master architecture, 146
on existing servers, 137-139
master and slave servers, 144-148
master, restarting, 136
new servers, 135-137
pyramid architecture, 147
replication ring (multi-master), 146
slave, restarting, 137
slave with two masters architecture, 145
files and settings, 142-144
filtering, 143
log files, 142
log index files, 142
status files, 143
implementation, differences in MySQL versions, 129
LOAD TABLE FROM MASTER command, 139
master and slave servers, 130
query processing, 140-142
over an intermittent connection, 130
performance, testing, 134
problems not solved by, 132-134
online ordering, 133
real time data transmission, 133
problems solved by, 130-132
backup and recovery using, 131
data distribution, 130
high availability and failover, 131
load-balancing using, 130
proposed enhancements, 164-168
fail-safe replication, 165
safe multi-master replication, 165-168
slaves, adding, 164
query processing, 140-142
MySQL 3.23, 140
MySQL 4.0, 141
rules, 144
slaves, copying master data to, 138
snapshots versus backups, 139
tools, 153-160
check_repl script, 154
fix_repl script, 155
mysqlbinlog utility, 153
mysqldiff script, 157
purge_binary_logs script, 156
read_heartbeat script, 159
write_heartbeat script, 159
requests, 171
RESET MASTER command, 152
REVOKE command, 215
role based addressing, 132
ROLLBACK statement, 22
round-robin DNS, 130
row locks, 20
R-tree indexes, 70-72
S[ Top ]
scripts, downloading, xii
SCSI versus IDE disks, 112
secondary indexes, 65
security, 203-236
accounts, 203
columns_priv table, 214
data encryption, 231-234
database-specific passwords, 204
db table, 212
encryption
application-level encryption, 233-234
encrypted filesystems, 232
hashing passwords, 231
general guidelines, 222
GRANT command, 215-219
grant tables, 204, 207
fields used for matching records in, 208
host table, 211
LOAD DATA INFILE... command, 206
logins, 203
MySQL in chroot environments, 235
networks, 223-231
automatic host blocking, 230
connection encryption and tunneling, 226-229
firewalls, 225
localhost and MySQL, 224
localhost-only connections, 223
MySQL in DMZs, 225
SSL, 227-229
TCP wrappers, 230
VPNs, 226
operating systems, 222
passwords, 210
privileges, 205-207
average employee accounts, 217
database administrator accounts, 216
logging, write-only access, 217
operations and monitoring personnel, 218
privilege checks, 207
revocation, limitations, 219
system administrator account, 215
problems and limitations, 219-221
host and database matching, 220
privileges and dropped databases, 220
revocation of specific privileges, 219
REVOKE command, 215
SHOW PROCESSLIST command, 206
source code modification, 235
tables_priv table, 213
user table, 208-211
host matching, 209
seek time, 104
select privilege, 206
serializable isolation level, 26
server ID, 136
server IDs, slaves, 144
server variables
displaying, 11
SESSION and GLOBAL syntax, 10
servers
hardware, buying, 107
health checks, 175-177
masters and slaves (see replication)
performance limiting factors, 103-109
CPU cache, 107
disks, 104
duplex mismatch, 106
I/O channels, 107
master and slaves replication setup, 106
memory, 105
memory speed, 107
networks, 106-109
redundant power supplies, 107
replication (see replication)
tuning, 119-128
bottlenecks identifying, 119
caching, 123
CPU bottlenecks, solving, 124
I/O bottlenecks, solving, 119
kernel bottlenecks, solving, 127
load-balancing, 123
memory needs, calculating, 125
queries using the wrong index, 120-122
temporary tables, 122
versus query optimization, 103
server-wide privileges, 207
SESSION syntax, 10
SET TRANSACTION ISOLATION LEVEL command, 28
SHA1( ), 231
shared locks, 18
SHOW commands, 11-15
SHOW CREATE TABLE, 77
SHOW INDEXES FROM, 77
SHOW INNODB STATUS, 14, 245-247
SHOW MASTER LOGS, 151
SHOW MASTER STATUS, 149
SHOW PROCESSLIST, 12-14
commands in output, 13
security concerns, 206
SHOW SLAVE STATUS, 150
SHOW STATUS, 14, 237-245
command counters, 239
data access patterns, 240
file descriptors, 242
handler counters, 240
MyISAM key buffer statistics counters, 241
query cache statistics counters, 242
SELECT queries, tracking, 243
sorts, 244
status counters, 237
table locking, 244
temporary files and tables counters, 239
thread and connection statistics counters, 238
SHOW TABLE STATUS, 35-37
SHOW VARIABLES, 11
shutdown privilege, 206
slave with two masters replication configuration, 145
slaves
connection to wrong master, 164
manual data changes to, 160
slow query log, 95
smack file, 54
soft updates, 114
software RAID, 112
spatial indexes, 70
SQL_BIG_RESULT hint, 97
SQL_BUFFER_RESULT hint, 97
SQL_CACHE hint, 81, 98
SQL_NO_CACHE hint, 80
SSH tunneling, 229
SSL (Secure Sockets Layer), 227-229
status files, 143, 175
stock quotes, use of storage engines for, 32
storage engines, 16, 35-44
Berkeley DB (BDB) storage engine, 44
features, 35
Heap (in-memory) tables, 44
InnoDB storage engine, 42-44
locking and concurrency, 43
referential integrity, 43
storage, 43
mixing in transactions, 28
MyISAM storage engine, 37-42
compressed tables, 39
Merge tables, 40-42
RAID tables, 39
storage, 37
NDB, 183
selecting, 29-34
backups, considerations, 30
special features, based on, 30
transactions and concurrency, considerations, 29
table conversions, 33
table types, determining, 35-37
transaction safe MySQL engines, 27
use, examples, 30
bulletin boards, 32
catalogs, 31
CD-ROM applications, 33
logging, 31
order processing, 31
stock quotes, 32
threaded discussion forums, 32
stress testing versus performance testing, 46
striping, 109
super privilege, 206
super-smack (see MySQL super-smack)
support-files directory, 7-10
swap, 117
systat, 124
system administrator accounts, database privileges for, 215
system versions, 21
T[ Top ]
table types (see storage engines)
tables
missing temporary tables, problems caused by, 163
reindexing, 77
table conversions, 33
ALTER TABLE statement, 33
CREATE and SELECT, 34
drop and reimport, 34
(see also storage engines)
tablespaces, 43
tables_priv table, 207, 213
TCP wrappers, 230
testing, backups and, 189
threaded discussion forums, use of storage engines for, 32
threading, 118
Time To Live (TTL), 131
tmp_table_size variable, 122
tools, downloading, xiv
top, 119
transaction logs, 27
transactions, 22
ACID transactions, 22-29
AUTOCOMMIT mode, 27
implicit commits, 27
isolation levels, setting, 28
simulating on non-transaction-safe tables, 29
storage engines, mixing in, 28
tunefs command (Linux), 116
Tuuri, Heikki, 42
U[ Top ]
UFS and UFS2 filesystems, 116
unique indexes, 65
versus primary keys, 67
Unix configuration file locations, 4
UNLOCK TABLES command, 29
user table, 207, 208-211
host matching, 209
V[ Top ]
Veritas, 186
filesystem snapshot tool, 199
versioning, 20
virtual private networks (VPNs), 226
vmstat, 119
VPNs (virtual private networks), 226
W[ Top ]
whitespace in queries, 80
wildcard matches and indexes, 75
Windows
benchmarking tools and, 49
binaries versus source installations, 2
configuration file locations, 4
journaling and non-journaling filesystems, 114
MySQL and, xii
Windows Task Manager, 119
write caching, 112
write locks, 18
write_heartbeat script, 159
Zurück zu High Performance MySQL