mysql学习记录（二十）
| 一、理论： 1.内存优化原则： a.将尽量多的内存分配给mysql做缓存，但要给操作系统和其他程序预留足够的内存，否则将产生SWAP页交换，影响自身性能 b.MyISAM的数据文件读取依赖于操作系统自身的IO缓存，因此，如果有MyISAM表，就要预留更多的内存给操作系统做IO缓存 c.排序区、连接区等缓存是分配给每个数据库会话专用的，其默认值的设置要根据最大连接数合理分配。不能设置太大否则在并发连接较高时会导致物理内存耗尽 2.MyISAM内存优化： a.key_buffer_size决定MyISAM索引块缓存区的大小，它直接影响MyISAM表的存取效率。可以在mysql参数文件中设置key_buffer_size的值，对于一般MyISAM的数据库建议至少将1/4可用内存分配给key_buffer_size key buffer使用率计算公式如下： 1 - ( (key_blocks_unused * key_cache_block_size) / key_buffer_size ) 在80%左右合适。大于80%将因索引缓存不足导致性能下降，小于80%会导致内存浪费 b.使用多个索引缓存： 1)mysql通过各session共享key buffer提高了MyISAM索引存取的性能，但它并不能消除session间key buffer间的竞争。 3.调整LRU策略： a.通过调节key_cache_division_limit来控制多大比例的缓存用做warm子表。 在最后： N * key_cache_age_threshold / 100次缓存命准内未被访问过，就会被降级到warm子表 b.调整read_buffer_size和read_md_buffer_size c.如带用order by子句，可以适当增大read_rnd_buffer_size的值。但read_rnd_buffer_size的值也是按session分配的 4.InnoDB内存优化： a.InnoDB用一块内存区做IO缓存池，该缓存池不仅用来缓存InnoDB的索引块，而且也用来缓存InnoDB的数据块。与MyISAM不同 b.在内部，InnoDB缓存池逻辑上由free list、flush list、LRU list组成。顾名思义，free list是空闲缓存块列表，flush list是需要缓新到磁盘的缓存块列表，而LRU list是InnoDB正在使用的缓存块，它是InnoDB buffer pool的核心。 InnoDB使用LRU算法与MyISAM的‘中点插入策略’LRU算法类似。 脏页的刷新 c.可以通过调整InnoDB buffer pool的大小，改变young sublist和old sublist的分配比例、控制脏缓存的刷新活动、使用多个InnoDB缓存池等方法来优化InnoDB的性能 5.innodb_buffer_pool_size的设置： a.innodb_buffer_pool_size决定InnoDB存储引擎表数据和索引数据的最大缓存区大小。innodb buffer pool同时为数据块和索引块提供数据缓存，若innodb_buffer_pool_size值越大，缓存命中率越高，访问InnoDB表需要的磁盘I/O就越少，性能也就越高。 可以通过： mysqladmin -S /tmp/mysql.sock ext | grep -i innodb_buffer_pool 查看buffer pool的使用情况 可用以下公式InnoDB缓存池的命中率： ( 1- innodb_buffer_pool_reads/innodb_buffer_pool_read_request) * 100 若太低则应该扩充内存、增加innodb_buffer_pool_size的值 6.调整innodb_old_blocks_time的设置： a.innodb_old_blocks_time参数决定了缓存数据块由old sublist转移到young sublist的快慢，当一个缓存数据块被插入到midpoint(old sublist)后，至少要在old sublist停留超过innodb_old_blocks_time(ms)后，才有可能被转移到new sublist. 可以根据InnoDB Monitor的输出信息来调整innodb_old_blocks_time的值，在进行表扫描时，如果non-youngs/s很低，young/s很高，就应考虑将innodb_old_blocks_time适当调大，以防止表扫描将真正的热数据淘汰，此值可以进行动态设置 7.调整缓存池数量，减少内部对缓存池数据结构的争用： a.InnoDB的缓存系统引入了innodb_buffer_pool_instances配置参数。对于较大的缓存池，适当增大此参数的值，可以降低并发导致的内部缓存访问冲突，改善性能。InnoDB缓存系统会将参数innodb_buffer_pool_size指定大小的缓存平分为innodb_buffer_pool_instances个buffer pool. 8.控制innodb buffer刷新，延长数据缓存时间，减缓磁盘I/O a.innodb buffer pool的刷新快慢主要取决于两个参数 1)innodb_max_dirty_pages_pct：控制缓存池中脏页的最大比例，默认值是75%，如果脏页的数量达到或超过该值，innoDB后台线程将开台缓存刷新 2)innodb_io_capacity：在某种程度上代表磁盘中每秒可完成的I/O次数，对于转速较低的磁盘，可将innodb_io_capacity降低。对于固态磁盘和由多个磁盘组成的阵列，innodb_io_capacity的值可适当增大 3)如无法增大缓存池，应将innodb_max_dirty_pages_pct的值调小，将innodb_io_capactity的值提高，以加快脏页的刷新 9.InnoDB doublewrite： a.在做恢复时，如果发现不一致的页，InnoDB会用系统表空间double buffer区的相应副本来恢复数据页 b.由于同步到doublewrite buffer是对连续磁盘空间的顺序写，因此开启双写对性能的影响并不太大。对需要高性能并且可以容忍丢失数据的应用，可将innodb_doublewrite=0来关闭双写以满足性能 10.调整用户服务线程排序缓存区： a.若查看show global status看到sort_merge_passes的值很大，则可以调整参数sort_buffer_size的值来调整排序缓存区，以改善order by子句或group子句的sql性能 b.可通过调整join_buffer_size的值来改善没使用索引的查询 c.最好的策略是设置较小的全局join_buffer_size,对较复杂的操作session单独设置join_buffer_size 11.InnoDB log机制及优化： a.innodb_flush_log_at_trx_commit的设置： 0：每秒触发一次，可满足持久化要求（效率最高，但最不安全） 1：每个事务提交时立刻将缓存中的redo日志回写到日志文件，并调用操作系统fsync刷新IO缓存（默认值，效率最低，但最安全） 2：每个事务提交时,InnoDB立刻将redo日志回写到日志文件，但并不马上调用fsync来刷新IO缓存，而是每秒只做一次磁盘IO缓存刷新操作（性能和数据安全性都在中间） 12.设置log file size,控制检查点： a.可以通过一些方法计算innodb每小时产生的日志量并估计合适的innodb_log_file_size值 13.调整innodb_log_buffer_size： a.可以通过增大innodb_log_buffer_size来减少日志写磁盘操作，从而提高事务处理的性能 14.调整max_connections，提高并发连接： a.max_connections控制允许连接到mysql数据库的最大数量，默认值是151 15.调整back_log： a.如果需要数据库在较短时间内处理大量连接请求，可适当增大back_log的值 16.调整table_open_cache： a.在未执行flush tables命令的情况下，如果mysql状态变量opened_tables的值较大，就说明table_open_cache设置的太小，应适当增大 17.调整thread_cache_size： a.可以通过计算cache的失败率threads_created/connections来衡量thread_cached_size的值是否合适。此值越接近1，说明线程cache命中率越低，应考虑适当增加thread_cache_size的值 18.innodb_lock_wait_timeout的设置： a.innodb_lock_wait_timeout可以控制innodb事务等待行锁的时间，默认值是50ms. b.对于需要快速反馈的交互式OLTP应用，可以将行锁等待超时时间调小，以避免事务长时间挂起 c.对于后台运行的批处理操作，可以将行锁等待超时时间调大，以避免发生大的回滚操作 二、实践：
mysql> use sakila;Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -ADatabase changedmysql> show engine innodb status /G;*************************** 1. row *************************** Type: InnoDB Name: Status: =====================================151102 7:13:01 INNODB MONITOR OUTPUT=====================================Per second averages calculated from the last 15 seconds-----------------BACKGROUND THREAD-----------------srv_master_thread loops: 113 1_second, 113 sleeps, 7 10_second, 47 background, 47 flushsrv_master_thread log flush and writes: 113----------SEMAPHORES----------OS WAIT ARRAY INFO: reservation count 18, signal count 18Mutex spin waits 8, rounds 240, OS waits 7RW-shared spins 11, rounds 330, OS waits 11RW-excl spins 0, rounds 0, OS waits 0Spin rounds per wait: 30.00 mutex, 30.00 RW-shared, 0.00 RW-excl------------TRANSACTIONS------------Trx id counter B0APurge done for trx's n:o < 920 undo n:o < 0History list length 103LIST OF TRANSACTIONS FOR EACH SESSION:---TRANSACTION 0, not startedMySQL thread id 28, OS thread handle 0x7f5dbdfe6700, query id 568 localhost rootshow engine innodb status--------FILE I/O--------I/O thread 0 state: waiting for i/o request (insert buffer thread)I/O thread 1 state: waiting for i/o request (log thread)I/O thread 2 state: waiting for i/o request (read thread)I/O thread 3 state: waiting for i/o request (read thread)I/O thread 4 state: waiting for i/o request (read thread)I/O thread 5 state: waiting for i/o request (read thread)I/O thread 6 state: waiting for i/o request (write thread)I/O thread 7 state: waiting for i/o request (write thread)I/O thread 8 state: waiting for i/o request (write thread)I/O thread 9 state: waiting for i/o request (write thread)Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0Pending flushes (fsync) log: 0; buffer pool: 0586 OS file reads, 55 OS file writes, 48 OS fsyncs0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s-------------------------------------INSERT BUFFER AND ADAPTIVE HASH INDEX-------------------------------------Ibuf: size 1, free list len 0, seg size 2, 0 mergesmerged operations: insert 0, delete mark 0, delete 0discarded operations: insert 0, delete mark 0, delete 0Hash table size 553229, node heap has 1 buffer(s)0.00 hash searches/s, 0.00 non-hash searches/s---LOG---Log sequence number 13476957Log flushed up to 13476957Last checkpoint at 134769570 pending log writes, 0 pending chkp writes32 log i/o's done, 0.00 log i/o's/second----------------------BUFFER POOL AND MEMORY----------------------Total memory allocated 274726912; in additional pool allocated 0Dictionary memory allocated 182244Buffer pool size 16383Free buffers 15829Database pages 553Old database pages 224Modified db pages 0Pending reads 0Pending writes: LRU 0, flush list 0, single page 0Pages made young 0, not young 00.00 youngs/s, 0.00 non-youngs/sPages read 553, created 0, written 270.00 reads/s, 0.00 creates/s, 0.00 writes/sNo buffer pool page gets since the last printoutPages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/sLRU len: 553, unzip_LRU len: 0I/O sum[0]:cur[0], unzip sum[0]:cur[0]--------------ROW OPERATIONS--------------0 queries inside InnoDB, 0 queries in queue1 read views open inside InnoDBMain thread process no. 62305, id 140040552920832, state: waiting for server activityNumber of rows inserted 5, updated 0, deleted 0, read 120.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s----------------------------END OF INNODB MONITOR OUTPUT============================1 row in set (0.00 sec)ERROR: No query specifiedmysql> set global hot_cache.key_buffer_size = 128* 1024;Query OK, 0 rows affected (0.00 sec)mysql> set global hot_cache.key_buffer_size = 0;Query OK, 0 rows affected (0.00 sec)mysql> show variables like 'key_buffer_size';+-----------------+----------+| Variable_name | Value |+-----------------+----------+| key_buffer_size | 33554432 |+-----------------+----------+1 row in set (0.00 sec)mysql> set global key_buffer_size = 0;ERROR 1438 (HY000): Cannot drop default keycachemysql> show warnings;+-------+------+------------------------------+| Level | Code | Message |+-------+------+------------------------------+| Error | 1438 | Cannot drop default keycache |+-------+------+------------------------------+1 row in set (0.00 sec)mysql> cache index sales,sales2 in hot_cache;+---------------+--------------------+----------+-------------------------------------+| Table | Op | Msg_type | Msg_text |+---------------+--------------------+----------+-------------------------------------+| sakila.sales | assign_to_keycache | Error | Table 'sakila.sales' doesn't exist || sakila.sales | assign_to_keycache | status | Operation failed || sakila.sales2 | assign_to_keycache | Error | Table 'sakila.sales2' doesn't exist || sakila.sales2 | assign_to_keycache | status | Operation failed |+---------------+--------------------+----------+-------------------------------------+4 rows in set (0.00 sec)mysql> set global key_cache_division_limit = 70;Query OK, 0 rows affected (0.00 sec)mysql> set global hot_cache.key_cache_division_limit = 70;Query OK, 0 rows affected (0.00 sec)mysql> show global variables like '%innodb_old_blocks_pct%';+-----------------------+-------+| Variable_name | Value |+-----------------------+-------+| innodb_old_blocks_pct | 37 |+-----------------------+-------+1 row in set (0.01 sec)mysql> show global variables like '%doublewrirte%';Empty set (0.00 sec)mysql> show global variables like '%doublewrite%';+--------------------+-------+| Variable_name | Value |+--------------------+-------+| innodb_doublewrite | ON |+--------------------+-------+1 row in set (0.01 sec)mysql> select @a1 := variable_value as a1;ERROR 1054 (42S22): Unknown column 'variable_value' in 'field list'mysql> select @a1 := variable_value as a1 -> from information_schema.global_status -> where variable_name = 'innodb_os_log_written' -> union all -> select sleep(60) -> union all -> select @a2 := variable_value as a2 -> from information_schema.global_status -> where variable_name = 'innodb_os_log_written';+------+| a1 |+------+| 9216 || 0 || 9216 |+------+3 rows in set (1 min 0.02 sec)mysql> select round((@a2-@a1)/1024/1024 / @@innodb_log_files_in_group) as MB;+------+| MB |+------+| 0 |+------+1 row in set (0.01 sec) |