7月 22
目标,千万级数据库。不免要考虑到如何拆表。
垂直拆表比较好处理,主要原则就是将定长字段与不定长字段分离。
水平拆表的逻辑会更复杂一些。优先考虑使用数据库内部实现。让数据库软件自己去承担表逻辑。
如果自己处理,通用的方式是永恒的二八原则:让80%的查询在小表,其他20%访问大表。

以下是关于MySQL partition的一些资料:

MySQL的LIST分区体验与总结
Mysql Forums :: Partition
MySQL 5.1 Reference Manual :: 21 Partitioning
通过分区(Partition)提升MySQL性能
Partitioning with Dates in MySQL 5.1


也有一个第三方的extensible用于水平拆分Mysql Table
HiveDB an Open Source framework for horizontally partitioning MySQL systems
Defined tags for this entry:

作者 rollenc

Bookmark MySQL拆表  at del.icio.us Digg MySQL拆表 Mixx MySQL拆表 Bloglines MySQL拆表 Technorati MySQL拆表 Fark this: MySQL拆表 Bookmark MySQL拆表  at YahooMyWeb Bookmark MySQL拆表  at Furl.net Bookmark MySQL拆表  at reddit.com Bookmark MySQL拆表  at blinklist.com Bookmark MySQL拆表  at Spurl.net Bookmark MySQL拆表  at NewsVine Bookmark MySQL拆表  at Simpy.com Bookmark MySQL拆表  at blogmarks Bookmark MySQL拆表  with wists Bookmark MySQL拆表  at Ma.gnolia.com wong it! Bookmark using any bookmark manager! Stumble It!
10月 13
今天升级mysql server遇到错误,错误信息如下:

Preparing to replace mysql-server-5.0 5.0.38-0ubuntu1 (using .../mysql-server-5.0_5.0.45-1ubuntu2_i386.deb) ...
* Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
dpkg: warning - old pre-removal script returned error exit status 1
dpkg - trying script from the new package instead ...
* Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
dpkg: error processing /var/cache/apt/archives/mysql-server-5.0_5.0.45-1ubuntu2_i386.deb (--unpack):
subprocess new pre-removal script returned error exit status 1
* Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
* Starting MySQL database server mysqld [ OK ]
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)'
Errors were encountered while processing:
/var/cache/apt/archives/mysql-server-5.0_5.0.45-1ubuntu2_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)


解决办法:

sudo cat /var/run/mysqld/mysqld.pid #获得mysql server的运行ID,填入下方kill后的 xxx
sudo kill xxx #'xxx'为上方的输出结果
 

再重新升级mysql server.

Defined tags for this entry: ,

作者 rollenc

Bookmark ubuntu升级Mysql Server失败的解决办法  at del.icio.us Digg ubuntu升级Mysql Server失败的解决办法 Mixx ubuntu升级Mysql Server失败的解决办法 Bloglines ubuntu升级Mysql Server失败的解决办法 Technorati ubuntu升级Mysql Server失败的解决办法 Fark this: ubuntu升级Mysql Server失败的解决办法 Bookmark ubuntu升级Mysql Server失败的解决办法  at YahooMyWeb Bookmark ubuntu升级Mysql Server失败的解决办法  at Furl.net Bookmark ubuntu升级Mysql Server失败的解决办法  at reddit.com Bookmark ubuntu升级Mysql Server失败的解决办法  at blinklist.com Bookmark ubuntu升级Mysql Server失败的解决办法  at Spurl.net Bookmark ubuntu升级Mysql Server失败的解决办法  at NewsVine Bookmark ubuntu升级Mysql Server失败的解决办法  at Simpy.com Bookmark ubuntu升级Mysql Server失败的解决办法  at blogmarks Bookmark ubuntu升级Mysql Server失败的解决办法  with wists Bookmark ubuntu升级Mysql Server失败的解决办法  at Ma.gnolia.com wong it! Bookmark using any bookmark manager! Stumble It!
4月 25
Mark 一下。
整理 描述
armscii8 (ARMSCII-8 Armenian)
armscii8_bin 亚美尼亚语, 二进制
armscii8_general_ci 亚美尼亚语, 不区分大小写
ascii (US ASCII)
ascii_bin 西欧 (多语言), 二进制
ascii_general_ci 西欧 (多语言), 不区分大小写
big5 (Big5 Traditional Chinese)
big5_bin 繁体中文, 二进制
big5_chinese_ci 繁体中文, 不区分大小写
binary (Binary pseudo charset)
binary 二进制
cp1250 (Windows Central European)
cp1250_bin 中欧 (多语言), 二进制
cp1250_croatian_ci 克罗地亚语, 不区分大小写
cp1250_czech_cs 捷克语, 区分大小写
cp1250_general_ci 中欧 (多语言), 不区分大小写
cp1251 (Windows Cyrillic)
cp1251_bin 西里尔语 (多语言), 二进制
cp1251_bulgarian_ci 保加利亚语, 不区分大小写
cp1251_general_ci 西里尔语 (多语言), 不区分大小写
cp1251_general_cs 西里尔语 (多语言), 区分大小写
cp1251_ukrainian_ci 乌克兰语, 不区分大小写
cp1256 (Windows Arabic)
cp1256_bin 阿拉伯语, 二进制
cp1256_general_ci 阿拉伯语, 不区分大小写
cp1257 (Windows Baltic)
cp1257_bin 巴拉克语 (多语言), 二进制
cp1257_general_ci 巴拉克语 (多语言), 不区分大小写
cp1257_lithuanian_ci 立陶宛语, 不区分大小写
cp850 (DOS West European)
cp850_bin 西欧 (多语言), 二进制
cp850_general_ci 西欧 (多语言), 不区分大小写
cp852 (DOS Central European)
cp852_bin 中欧 (多语言), 二进制
cp852_general_ci 中欧 (多语言), 不区分大小写
cp866 (DOS Russian)
cp866_bin 俄语, 二进制
cp866_general_ci 俄语, 不区分大小写
cp932 (SJIS for Windows Japanese)
cp932_bin 日语, 二进制
cp932_japanese_ci 日语, 不区分大小写
dec8 (DEC West European)
dec8_bin 西欧 (多语言), 二进制
dec8_swedish_ci 瑞典语, 不区分大小写
eucjpms (UJIS for Windows Japanese)
eucjpms_bin 日语, 二进制
eucjpms_japanese_ci 日语, 不区分大小写
euckr (EUC-KR Korean)
euckr_bin 朝鲜语, 二进制
euckr_korean_ci 朝鲜语, 不区分大小写
gb2312 (GB2312 Simplified Chinese)
gb2312_bin 简体中文, 二进制
gb2312_chinese_ci 简体中文, 不区分大小写
gbk (GBK Simplified Chinese)
gbk_bin 简体中文, 二进制
gbk_chinese_ci 简体中文, 不区分大小写
geostd8 (GEOSTD8 Georgian)
geostd8_bin 乔治亚语, 二进制
geostd8_general_ci 乔治亚语, 不区分大小写
greek (ISO 8859-7 Greek)
greek_bin 希腊语, 二进制
greek_general_ci 希腊语, 不区分大小写
hebrew (ISO 8859-8 Hebrew)
hebrew_bin 希伯来语, 二进制
hebrew_general_ci 希伯来语, 不区分大小写
hp8 (HP West European)
hp8_bin 西欧 (多语言), 二进制
hp8_english_ci 英语, 不区分大小写
keybcs2 (DOS Kamenicky Czech-Slovak)
keybcs2_bin 捷克斯洛伐克语, 二进制
keybcs2_general_ci 捷克斯洛伐克语, 不区分大小写
koi8r (KOI8-R Relcom Russian)
koi8r_bin 俄语, 二进制
koi8r_general_ci 俄语, 不区分大小写
koi8u (KOI8-U Ukrainian)
koi8u_bin 乌克兰语, 二进制
koi8u_general_ci 乌克兰语, 不区分大小写
latin1 (cp1252 West European)
latin1_bin 西欧 (多语言), 二进制
latin1_danish_ci 丹麦语, 不区分大小写
latin1_general_ci 西欧 (多语言), 不区分大小写
latin1_general_cs 西欧 (多语言), 区分大小写
latin1_german1_ci 德语 (字典), 不区分大小写
latin1_german2_ci 德语 (电话本), 不区分大小写
latin1_spanish_ci 西班牙语, 不区分大小写
latin1_swedish_ci 瑞典语, 不区分大小写

整理 描述
latin2 (ISO 8859-2 Central European)
latin2_bin 中欧 (多语言), 二进制
latin2_croatian_ci 克罗地亚语, 不区分大小写
latin2_czech_cs 捷克语, 区分大小写
latin2_general_ci 中欧 (多语言), 不区分大小写
latin2_hungarian_ci 匈牙利语, 不区分大小写
latin5 (ISO 8859-9 Turkish)
latin5_bin 土耳其语, 二进制
latin5_turkish_ci 土耳其语, 不区分大小写
latin7 (ISO 8859-13 Baltic)
latin7_bin 巴拉克语 (多语言), 二进制
latin7_estonian_cs 爱沙尼亚语, 区分大小写
latin7_general_ci 巴拉克语 (多语言), 不区分大小写
latin7_general_cs 巴拉克语 (多语言), 区分大小写
macce (Mac Central European)
macce_bin 中欧 (多语言), 二进制
macce_general_ci 中欧 (多语言), 不区分大小写
macroman (Mac West European)
macroman_bin 西欧 (多语言), 二进制
macroman_general_ci 西欧 (多语言), 不区分大小写
sjis (Shift-JIS Japanese)
sjis_bin 日语, 二进制
sjis_japanese_ci 日语, 不区分大小写
swe7 (7bit Swedish)
swe7_bin 瑞典语, 二进制
swe7_swedish_ci 瑞典语, 不区分大小写
tis620 (TIS620 Thai)
tis620_bin 泰语, 二进制
tis620_thai_ci 泰语, 不区分大小写
ucs2 (UCS-2 Unicode)
ucs2_bin Unicode (多语言), 二进制
ucs2_czech_ci 捷克语, 不区分大小写
ucs2_danish_ci 丹麦语, 不区分大小写
ucs2_esperanto_ci Esperanto, 不区分大小写
ucs2_estonian_ci 爱沙尼亚语, 不区分大小写
ucs2_general_ci Unicode (多语言), 不区分大小写
ucs2_hungarian_ci 匈牙利语, 不区分大小写
ucs2_icelandic_ci 冰岛语, 不区分大小写
ucs2_latvian_ci 拉脱维亚语, 不区分大小写
ucs2_lithuanian_ci 立陶宛语, 不区分大小写
ucs2_persian_ci 波斯语, 不区分大小写
ucs2_polish_ci 波兰语, 不区分大小写
ucs2_roman_ci 西欧, 不区分大小写
ucs2_romanian_ci 罗马尼亚语, 不区分大小写
ucs2_slovak_ci 斯洛伐克语, 不区分大小写
ucs2_slovenian_ci 斯洛文尼亚语, 不区分大小写
ucs2_spanish2_ci 传统西班牙语, 不区分大小写
ucs2_spanish_ci 西班牙语, 不区分大小写
ucs2_swedish_ci 瑞典语, 不区分大小写
ucs2_turkish_ci 土耳其语, 不区分大小写
ucs2_unicode_ci Unicode (多语言), 不区分大小写
ujis (EUC-JP Japanese)
ujis_bin 日语, 二进制
ujis_japanese_ci 日语, 不区分大小写
utf8 (UTF-8 Unicode)
utf8_bin Unicode (多语言), 二进制
utf8_czech_ci 捷克语, 不区分大小写
utf8_danish_ci 丹麦语, 不区分大小写
utf8_esperanto_ci Esperanto, 不区分大小写
utf8_estonian_ci 爱沙尼亚语, 不区分大小写
utf8_general_ci Unicode (多语言), 不区分大小写
utf8_hungarian_ci 匈牙利语, 不区分大小写
utf8_icelandic_ci 冰岛语, 不区分大小写
utf8_latvian_ci 拉脱维亚语, 不区分大小写
utf8_lithuanian_ci 立陶宛语, 不区分大小写
utf8_persian_ci 波斯语, 不区分大小写
utf8_polish_ci 波兰语, 不区分大小写
utf8_roman_ci 西欧, 不区分大小写
utf8_romanian_ci 罗马尼亚语, 不区分大小写
utf8_slovak_ci 斯洛伐克语, 不区分大小写
utf8_slovenian_ci 斯洛文尼亚语, 不区分大小写
utf8_spanish2_ci 传统西班牙语, 不区分大小写
utf8_spanish_ci 西班牙语, 不区分大小写
utf8_swedish_ci 瑞典语, 不区分大小写
utf8_turkish_ci 土耳其语, 不区分大小写
utf8_unicode_ci Unicode (多语言), 不区分大小写
Defined tags for this entry: ,

作者 rollenc

Bookmark 恼人的MySQL字符集和整理  at del.icio.us Digg 恼人的MySQL字符集和整理 Mixx 恼人的MySQL字符集和整理 Bloglines 恼人的MySQL字符集和整理 Technorati 恼人的MySQL字符集和整理 Fark this: 恼人的MySQL字符集和整理 Bookmark 恼人的MySQL字符集和整理  at YahooMyWeb Bookmark 恼人的MySQL字符集和整理  at Furl.net Bookmark 恼人的MySQL字符集和整理  at reddit.com Bookmark 恼人的MySQL字符集和整理  at blinklist.com Bookmark 恼人的MySQL字符集和整理  at Spurl.net Bookmark 恼人的MySQL字符集和整理  at NewsVine Bookmark 恼人的MySQL字符集和整理  at Simpy.com Bookmark 恼人的MySQL字符集和整理  at blogmarks Bookmark 恼人的MySQL字符集和整理  with wists Bookmark 恼人的MySQL字符集和整理  at Ma.gnolia.com wong it! Bookmark using any bookmark manager! Stumble It!
4月 24
这篇文章是mysql的交互式文章。目前并不足1000条,将来也不止1000条。需要大家努力。
原文见:http://forge.mysql.com/wiki/Top10SQLPerformanceTips

Specific Query Performance Tips (see also database design tips for tips on indexes):

1. Use EXPLAIN to profile the query execution plan
2. Use Slow Query Log (always have it on!)
3. Don't use DISTINCT when you have or could use GROUP BY
4. Insert performance
1. Batch INSERT and REPLACE
2. Use LOAD DATA instead of INSERT
5. LIMIT m,n may not be as fast as it sounds
6. Don't use ORDER BY RAND() if you have > ~2K records
7. Use SQL_NO_CACHE when you are SELECTing frequently updated data or large sets of data
8. Avoid wildcards at the start of LIKE queries
9. Avoid correlated subqueries and in select and where clause (try to avoid in)
10. No calculated comparisons -- isolate indexed columns
11. ORDER BY and LIMIT work best with equalities and covered indexes
12. Separate text/blobs from metadata, don't put text/blobs in results if you don't need them
13. Derived tables (subqueries in the FROM clause) can be useful for retrieving BLOBs without sorting them. (Self-join can speed up a query if 1st part finds the IDs and uses then to fetch the rest)
14. ALTER TABLE...ORDER BY can take data sorted chronologically and re-order it by a different field -- this can make queries on that field run faster (maybe this goes in indexing?)
15. Know when to split a complex query and join smaller ones
16. Delete small amounts at a time if you can
17. Make similar queries consistent so cache is used
18. Have good SQL query standards
19. Don't use deprecated features
20. Turning OR on multiple index fields (<5.0) into UNION may speed things up (with LIMIT), after 5.0 the index_merge should pick stuff up.
21. Don't use COUNT * on Innodb tables for every search, do it a few times and/or summary tables, or if you need it for the total # of rows, use SQL_CALC_FOUND_ROWS and SELECT FOUND_ROWS()
22. Use INSERT ... ON DUPLICATE KEY update (INSERT IGNORE) to avoid having to SELECT
23. use groupwise maximum instead of subqueries


Scaling Performance Tips:

1. Use benchmarking
2. isolate workloads don't let administrative work interfere with customer performance. (ie backups)
3. Debugging sucks, testing rocks!
4. As your data grows, indexing may change (cardinality and selectivity change). Structuring may want to change. Make your schema as modular as your code. Make your code able to scale. Plan and embrace change, and get developers to do the same.

Network Performance Tips:

1. Minimize traffic by fetching only what you need.
1. Paging/chunked data retrieval to limit
2. Don't use SELECT *
3. Be wary of lots of small quick queries if a longer query can be more efficient
2. Use multi_query if appropriate to reduce round-trips

OS Performance Tips:

1. Use proper data partitions
1. For Cluster. Start thinking about Cluster *before* you need them
2. Keep the database host as clean as possible. Do you really need a windowing system on that server?
3. Utilize the strengths of the OS
4. pare down cron scripts
5. create a test environment
6. source control schema and config files
7. for LVM innodb backups, restore to a different instance of MySQL so Innodb can roll forward
8. partition appropriately
9. partition your database when you have real data -- do not assume you know your dataset until you have real data

MySQL Server Overall Tips:

1. innodb_flush_commit=0 can help slave lag
2. Optimize for data types, use consistent data types. Use PROCEDURE ANALYSE() to help determine the smallest data type for your needs.
3. use optimistic locking, not pessimistic locking. try to use shared lock, not exclusive lock. share mode vs. FOR UPDATE
4. if you can, compress text/blobs
5. compress static data
6. don't back up static data as often
7. enable and increase the query and buffer caches if appropriate
8. config params -- http://docs.cellblue.nl/2007/03/17/easy-mysql-performance-tweaks/ is a good reference
9. Config variables & tips:
1. use one of the supplied config files
2. key_buffer, unix cache (leave some RAM free), per-connection variables, innodb memory variables
3. be aware of global vs. per-connection variables
4. check SHOW STATUS and SHOW VARIABLES (GLOBAL|SESSION in 5.0 and up)
5. be aware of swapping esp. with Linux, "swappiness" (bypass OS filecache for innodb data files, innodb_flush_method=O_DIRECT if possible (this is also OS specific))
6. defragment tables, rebuild indexes, do table maintenance
7. If you use innodb_flush_txn_commit=1, use a battery-backed hardware cache write controller
8. more RAM is good so faster disk speed
9. use 64-bit architectures
10. --skip-name-resolve
11. increase myisam_sort_buffer_size to optimize large inserts (this is a per-connection variable)
12. look up memory tuning parameter for on-insert caching
13. increase temp table size in a data warehousing environment (default is 32Mb) so it doesn't write to disk (also constrained by max_heap_table_size, default 16Mb)
14. Run in SQL_MODE=STRICT to help identify warnings
15. /tmp dir on battery-backed write cache
16. consider battery-backed RAM for innodb logfiles
17. use --safe-updates for client
18. Redundant data is redundant

Storage Engine Performance Tips:

1. InnoDB ALWAYS keeps the primary key as part of each index, so do not make the primary key very large
2. Utilize different storage engines on master/slave ie, if you need fulltext indexing on a table.
3. BLACKHOLE engine and replication is much faster than FEDERATED tables for things like logs.
4. Know your storage engines and what performs best for your needs, know that different ones exist.
1. ie, use MERGE tables ARCHIVE tables for logs
2. Archive old data -- don't be a pack-rat! 2 common engines for this are ARCHIVE tables and MERGE tables
5. use row-level instead of table-level locking for OLTP workloads
6. try out a few schemas and storage engines in your test environment before picking one.

Database Design Performance Tips:

1. Design sane query schemas. don't be afraid of table joins, often they are faster than denormalization
2. Don't use boolean flags
3. Use Indexes
4. Don't Index Everything
5. Do not duplicate indexes
6. Do not use large columns in indexes if the ratio of SELECTs:INSERTs is low.
7. be careful of redundant columns in an index or across indexes
8. Use a clever key and ORDER BY instead of MAX
9. Normalize first, and denormalize where appropriate.
10. Databases are not spreadsheets, even though Access really really looks like one. Then again, Access isn't a real database
11. use INET_ATON and INET_NTOA for IP addresses, not char or varchar
12. make it a habit to REVERSE() email addresses, so you can easily search domains (this will help avoid wildcards at the start of LIKE queries if you want to find everyone whose e-mail is in a certain domain)
13. A NULL data type can take more room to store than NOT NULL
14. Choose appropriate character sets & collations -- UTF16 will store each character in 2 bytes, whether it needs it or not, latin1 is faster than UTF8.
15. Use Triggers wisely
16. use min_rows and max_rows to specify approximate data size so space can be pre-allocated and reference points can be calculated.
17. Use HASH indexing for indexing across columns with similar data prefixes
18. Use myisam_pack_keys for int data
19. be able to change your schema without ruining functionality of your code
20. segregate tables/databases that benefit from different configuration variables

Other:

1. Hire a MySQL (tm) Certified DBA
2. Know that there are many consulting companies out there that can help, as well as MySQL's Professional Services.
3. Read and post to MySQL Planet at http://www.mysqlplanet.org
4. Attend the yearly MySQL Conference and Expo or other conferences with MySQL tracks (link to the conference here)
5. Support your local User Group (link to forge page w/user groups here)
Defined tags for this entry:

作者 rollenc

Bookmark 好文章:Top 1000 SQL Performance Tips  at del.icio.us Digg 好文章:Top 1000 SQL Performance Tips Mixx 好文章:Top 1000 SQL Performance Tips Bloglines 好文章:Top 1000 SQL Performance Tips Technorati 好文章:Top 1000 SQL Performance Tips Fark this: 好文章:Top 1000 SQL Performance Tips Bookmark 好文章:Top 1000 SQL Performance Tips  at YahooMyWeb Bookmark 好文章:Top 1000 SQL Performance Tips  at Furl.net Bookmark 好文章:Top 1000 SQL Performance Tips  at reddit.com Bookmark 好文章:Top 1000 SQL Performance Tips  at blinklist.com Bookmark 好文章:Top 1000 SQL Performance Tips  at Spurl.net Bookmark 好文章:Top 1000 SQL Performance Tips  at NewsVine Bookmark 好文章:Top 1000 SQL Performance Tips  at Simpy.com Bookmark 好文章:Top 1000 SQL Performance Tips  at blogmarks Bookmark 好文章:Top 1000 SQL Performance Tips  with wists Bookmark 好文章:Top 1000 SQL Performance Tips  at Ma.gnolia.com wong it! Bookmark using any bookmark manager! Stumble It!
3月 21
PHP 手册
Smarty - the compiling PHP template engine
MySQL 5.1参考手册
JavaScript Reference
prototype.js
Apache HTTP Server Version 2.2 Documentation
Bash Reference Manual
Subversion
Defined tags for this entry: , , , ,

作者 rollenc

Bookmark Manuals online  at del.icio.us Digg Manuals online Mixx Manuals online Bloglines Manuals online Technorati Manuals online Fark this: Manuals online Bookmark Manuals online  at YahooMyWeb Bookmark Manuals online  at Furl.net Bookmark Manuals online  at reddit.com Bookmark Manuals online  at blinklist.com Bookmark Manuals online  at Spurl.net Bookmark Manuals online  at NewsVine Bookmark Manuals online  at Simpy.com Bookmark Manuals online  at blogmarks Bookmark Manuals online  with wists Bookmark Manuals online  at Ma.gnolia.com wong it! Bookmark using any bookmark manager! Stumble It!