# 《Mycat核心技术》第10章：按日期(天)分片
作者：冰河
星球：http://m6z.cn/6aeFbs (opens new window)
博客：https://binghe.gitcode.host (opens new window)
文章汇总：https://binghe.gitcode.host/md/all/all.html (opens new window)
星球项目地址：https://binghe.gitcode.host/md/zsxq/introduce.html (opens new window)
沉淀，成长，突破，帮助他人，成就自我。
- 本章难度：★★☆☆☆
- 本章重点：介绍Mycat按日期(天)分片，掌握Mycat按日期(天)分片的核心原理与算法，能够结合自身实际项目将Mycat按日期(天)分片灵活应用到自身实际项目中。
大家好，我是CurleyG~~
今天给大家介绍《Mycat核心技术》的第10章：给大家简单介绍下Mycat按日期(天)分片，好了，开始今天的内容。
# 一、简单描述
这篇文章就给大家介绍如何用Mycat实现按日期(天)来进行数据分片操作，
即满足如下要求：
- 按日期（天）分片：从开始日期算起，按照天数来分片。
- 例如，从2017-01-01，每10天一个分片。
- 注意事项：需要提前将分片规划好，建好，否则有可能日期超出实际配置分片数。
# 二、实现数据分片
# 2.1 配置rule.xml
在rule.xml中添加如下配置：
<tableRule name="sharding-by-date-customer">
<rule>
<columns>create_time</columns>
<algorithm>sharding-by-date</algorithm>
</rule>
</tableRule>
<function name="sharding-by-date" class="org.opencloudb.route.function.PartitionByDate">
<property name="dateFormat">yyyy-MM-dd</property> <!--日期格式-->
<property name="sBeginDate">2017-01-01</property> <!--开始日期-->
<property name="sPartionDay">10</property> <!--每分片天数-->
</function>
1
2
3
4
5
6
7
8
9
10
11
12
2
3
4
5
6
7
8
9
10
11
12
# 2.2 配置schema.xml
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://org.opencloudb/" >
<schema name="lyzdb" checkSQLschema="false" sqlMaxLimit="100">
<!-- global table is auto cloned to all defined data nodes ,so can join
with any table whose sharding node is in the same data node -->
<table name="t_order" primaryKey="id" dataNode="dn1,dn2" rule="sharding-by-date-customer"/>
</schema>
<!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743"
/> -->
<dataNode name="dn1" dataHost="localhost1" database="db1" />
<dataNode name="dn2" dataHost="localhost1" database="db2" />
<!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
<dataNode name="jdbc_dn1" dataHost="jdbchost" database="db1" />
<dataNode name="jdbc_dn2" dataHost="jdbchost" database="db2" />
<dataNode name="jdbc_dn3" dataHost="jdbchost" database="db3" /> -->
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="192.168.209.137:3306" user="root" password="root"></writeHost>
<!--<writeHost host="hostS1" url="localhost:3316" user="root"-->
<!--password="123456" />-->
<!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
</dataHost>
</mycat:schema>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
至此，Mycat的配置工作就完成了，是不是很简单呢，下面我们就通过测试来分析一下Mycat的路由规则。
# 三、测试数据分片
# 3.1 建表并录入数据
首先，我们执行建表操作
mysql> explain CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME);
+-----------+----------------------------------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+----------------------------------------------------------------------------------------------------------------------------+
| dn1 | CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME) |
| dn2 | CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME) |
+-----------+----------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.01 sec)
mysql>
mysql>
mysql> CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME);
Query OK, 0 rows affected (0.27 sec)
1
2
3
4
5
6
7
8
9
10
11
12
13
2
3
4
5
6
7
8
9
10
11
12
13
可以看出，建表SQL路由到了所有的数据节点上。
接下来我们执行数据录入操作
mysql> explain INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09');
+-----------+-----------------------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+-----------------------------------------------------------------------------------------------------------------+
| dn1 | INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09') |
+-----------+-----------------------------------------------------------------------------------------------------------------+
1 row in set (0.05 sec)
mysql> INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09');
Query OK, 1 row affected (0.05 sec)
mysql> explain INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11');
+-----------+-----------------------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+-----------------------------------------------------------------------------------------------------------------+
| dn2 | INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11') |
+-----------+-----------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql> INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11');
Query OK, 1 row affected (0.01 sec)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
我们可以看出，日期为2017-01-09的数据SQL路由到了dn1节点，日期为2017-01-11的数据路由到了dn2节点，符合我们配置的Mycat路由规则。
# 3.2 录入数据的Mycat日志
我们打开Mycat的日志，观察如下：
08/09 23:23:03.272 DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]explain INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09')
08/09 23:23:07.718 DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09')
08/09 23:23:07.719 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09'), route={
1 -> dn1{INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'beijing','beijing10001_2017-01-09','2017-01-09')}
} rrs
08/09 23:23:59.154 DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]explain INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11')
08/09 23:23:59.341 DEBUG [Timer0] (SQLJob.java:85) -con query sql:select user() to con:MySQLConnection [id=3, lastTime=1502292239341, user=root, schema=db1, old shema=db1, borrowed=true, fromSlaveDB=false, threadId=49, charset=utf8, txIsolation=0, autocommit=true, attachment=null, respHandler=null, host=192.168.209.137, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
08/09 23:23:59.342 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=3, lastTime=1502292239329, user=root, schema=db1, old shema=db1, borrowed=true, fromSlaveDB=false, threadId=49, charset=utf8, txIsolation=0, autocommit=true, attachment=null, respHandler=null, host=192.168.209.137, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
08/09 23:24:02.624 DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11')
08/09 23:24:02.625 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11'), route={
1 -> dn2{INSERT INTO t_order (ID,PROVINCE,SN,CREATE_TIME) VALUES(20002,'beijing','beijing20002_2017-01-11','2017-01-11')}
} rrs
1
2
3
4
5
6
7
8
9
10
11
12
13
2
3
4
5
6
7
8
9
10
11
12
13
从日志我们也可以分析：出日期为2017-01-09的数据SQL路由到了dn1节点，日期为2017-01-11的数据路由到了dn2节点，符合我们配置的Mycat路由规则。
# 3.3 指定分片字段查询
mysql> explain select * from t_order where create_time = '2017-01-09';
+-----------+------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+------------------------------------------------------------------+
| dn1 | SELECT * FROM t_order WHERE create_time = '2017-01-09' LIMIT 100 |
+-----------+------------------------------------------------------------------+
1 row in set (0.21 sec)
1
2
3
4
5
6
7
2
3
4
5
6
7
可以看出： 执行简单的查询，如果指定分片字段ID字段，则走分片查询单个分片节点 查看后台mycat日志信息：
08/09 23:28:34.883 DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]explain select * from t_order where create_time = '2017-01-09'
08/09 23:28:34.909 DEBUG [$_NIOREACTOR-0-RW] (EnchachePool.java:76) -SQLRouteCache miss cache ,key:lyzdb select * from t_order where create_time = '2017-01-09'
08/09 23:28:35.094 DEBUG [$_NIOREACTOR-0-RW] (EnchachePool.java:59) -SQLRouteCache add cache ,key:lyzdb select * from t_order where create_time = '2017-01-09' value:select * from t_order where create_time = '2017-01-09', route={
1 -> dn1{SELECT *
FROM t_order
WHERE create_time = '2017-01-09'
LIMIT 100}
1
2
3
4
5
6
7
2
3
4
5
6
7
# 3.4 按照分片字段范围查询
mysql> explain select * from t_order where create_time < '2017-01-10';
+-----------+------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+------------------------------------------------------------------+
| dn1 | SELECT * FROM t_order WHERE create_time < '2017-01-10' LIMIT 100 |
| dn2 | SELECT * FROM t_order WHERE create_time < '2017-01-10' LIMIT 100 |
+-----------+------------------------------------------------------------------+
2 rows in set (0.00 sec)
1
2
3
4
5
6
7
8
2
3
4
5
6
7
8
可以得出：如果分片字段范围的查询，则走所有节点去检索，哪怕只有一条数据在一个分片上，route路由也是走所有的分片进行检索查询
# 3.5 按照非分片字段查询
mysql> explain select * from t_order where sn = 'beijing10001_2017-01-09';
+-----------+----------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+----------------------------------------------------------------------+
| dn1 | SELECT * FROM t_order WHERE sn = 'beijing10001_2017-01-09' LIMIT 100 |
| dn2 | SELECT * FROM t_order WHERE sn = 'beijing10001_2017-01-09' LIMIT 100 |
+-----------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)
1
2
3
4
5
6
7
8
2
3
4
5
6
7
8
可以得出：如果不走分片字段的查询，即使是单个数据，也要route路由所有的分片，走所有的分片进行查询。
