程序员最近都爱上了这个网站  程序员们快来瞅瞅吧!  it98k网:it98k.com

本站消息

站长简介/公众号

  出租广告位,需要合作请联系站长


+关注
已关注

分类  

暂无分类

标签  

暂无标签

日期归档  

2023-06(2)

【读写分离】SpringBoot整合多数据源实现读写分离(一)

发布于2021-03-07 20:28     阅读(764)     评论(0)     点赞(24)     收藏(4)


目录

背景

分析

准备工作

编码

1、多数据源配置文件

2、DataSource配置

3、MybatisConfig配置

4、创建 ClientDataSource 枚举 定义主从库

5、创建 ClientDataSourceContextHolder 来保存 ClientDataSource

6、继承 AbstractRoutingDataSource

7、通过注解 DataSourceRouting 来标识走master/slave

8、创建 DataSourceRoutingAspect,来处理注解 DataSourceRouting

9、主类 ReadWriteSeparationMybatisApplication

10、测试

TestController:

 UserServiceImpl

UserDAO

10、总结

11、源码


背景

实际项目中大都读多写少,如果查询出现瓶颈之后,我们可以考虑使用读写分离。

比如有三台Mysql服务器A、B、C,一主二从,先配置好 主从复制 之后,再来做读写分离,A用来做update操作,B和C用来做select操作。

网上很多文章都写的比较乱,这里我尽量简单优雅的完成。

分析

有很多中间件可以使用,比如:Mycat,当当的Sharding-JDBC,美团的DBProxy等,但是都需要依赖第三方组件,增加学习和money成本,

这里我们使用Spring提供的轻量级数据路由类 AbstractRoutingDataSource 来实现

准备工作

1、我这边准备两个DB,maple_master,maple_slave,主从复制这里就不做了,感兴趣的可以看这里主从复制 ;

DDL和DML为:

  1. CREATE TABLE `user` (
  2. `user_id` varchar(16) NOT NULL,
  3. `user_name` varchar(64) DEFAULT NULL,
  4. PRIMARY KEY (`user_id`)
  5. ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='user';
  6. INSERT INTO `maple_master`.`user`(`user_id`, `user_name`) VALUES ('1', 'maple_master');
  7. INSERT INTO `maple_slave`.`user`(`user_id`, `user_name`) VALUES ('1', 'maple_slave');

2、我这里使用常规的技术栈:SpingBoot + Mybatis + Maven,首先导入pom和基本编码

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3. xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  4. <modelVersion>4.0.0</modelVersion>
  5. <parent>
  6. <groupId>org.springframework.boot</groupId>
  7. <artifactId>spring-boot-starter-parent</artifactId>
  8. <version>2.4.3</version>
  9. <relativePath/> <!-- lookup parent from repository -->
  10. </parent>
  11. <groupId>com.gane.maple</groupId>
  12. <artifactId>read-write-separation-mybatisplus</artifactId>
  13. <version>0.0.1-SNAPSHOT</version>
  14. <name>read-write-separation-mybatis</name>
  15. <description>Demo project for Spring Boot</description>
  16. <properties>
  17. <java.version>1.8</java.version>
  18. </properties>
  19. <dependencies>
  20. <dependency>
  21. <groupId>org.springframework.boot</groupId>
  22. <artifactId>spring-boot-starter</artifactId>
  23. </dependency>
  24. <dependency>
  25. <groupId>org.springframework.boot</groupId>
  26. <artifactId>spring-boot-starter-web</artifactId>
  27. </dependency>
  28. <dependency>
  29. <groupId>org.springframework.boot</groupId>
  30. <artifactId>spring-boot-starter-aop</artifactId>
  31. <version>2.4.2</version>
  32. </dependency>
  33. <dependency>
  34. <groupId>mysql</groupId>
  35. <artifactId>mysql-connector-java</artifactId>
  36. <scope>runtime</scope>
  37. </dependency>
  38. <dependency>
  39. <groupId>com.alibaba</groupId>
  40. <artifactId>druid-spring-boot-starter</artifactId>
  41. <version>1.1.10</version>
  42. </dependency>
  43. <dependency>
  44. <groupId>org.springframework</groupId>
  45. <artifactId>spring-jdbc</artifactId>
  46. </dependency>
  47. <dependency>
  48. <groupId>org.mybatis.spring.boot</groupId>
  49. <artifactId>mybatis-spring-boot-starter</artifactId>
  50. <version>2.1.3</version>
  51. </dependency>
  52. <dependency>
  53. <groupId>org.apache.commons</groupId>
  54. <artifactId>commons-lang3</artifactId>
  55. <version>3.9</version>
  56. </dependency>
  57. <dependency>
  58. <groupId>org.projectlombok</groupId>
  59. <artifactId>lombok</artifactId>
  60. <optional>true</optional>
  61. </dependency>
  62. <dependency>
  63. <groupId>org.springframework.boot</groupId>
  64. <artifactId>spring-boot-starter-test</artifactId>
  65. <scope>test</scope>
  66. </dependency>
  67. </dependencies>
  68. <build>
  69. <plugins>
  70. <plugin>
  71. <groupId>org.springframework.boot</groupId>
  72. <artifactId>spring-boot-maven-plugin</artifactId>
  73. </plugin>
  74. </plugins>
  75. </build>
  76. </project>

编码

1、多数据源配置文件

  1. #master
  2. spring.datasource.master.type=com.alibaba.druid.pool.DruidDataSource
  3. spring.datasource.master.driver-class-name=com.mysql.cj.jdbc.Driver
  4. spring.datasource.master.jdbc-url=jdbc:mysql://localhost:3306/maple_master?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&zeroDateTimeBehavior=convertToNull
  5. spring.datasource.master.username=root
  6. spring.datasource.master.password=root
  7. #slave
  8. spring.datasource.slave.type=com.alibaba.druid.pool.DruidDataSource
  9. spring.datasource.slave.driver-class-name=com.mysql.cj.jdbc.Driver
  10. spring.datasource.slave.jdbc-url=jdbc:mysql://localhost:3306/maple_slave?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&zeroDateTimeBehavior=convertToNull
  11. spring.datasource.slave.username=root
  12. spring.datasource.slave.password=root

2、DataSource配置

  1. package com.gane.maple.jdbc.datasource;
  2. import com.gane.maple.jdbc.routing.ClientDataSource;
  3. import com.gane.maple.jdbc.routing.component.ClientDataSourceRouter;
  4. import org.springframework.beans.factory.annotation.Qualifier;
  5. import org.springframework.boot.context.properties.ConfigurationProperties;
  6. import org.springframework.boot.jdbc.DataSourceBuilder;
  7. import org.springframework.context.annotation.Bean;
  8. import org.springframework.context.annotation.Configuration;
  9. import org.springframework.context.annotation.Primary;
  10. import javax.sql.DataSource;
  11. import java.util.HashMap;
  12. import java.util.Map;
  13. /**
  14. * @author maple
  15. * @date 2021/3/3
  16. */
  17. @Configuration
  18. public class DataSourceConfig {
  19. @Bean(name = "masterDataSource")
  20. @ConfigurationProperties(prefix = "spring.datasource.master")
  21. public DataSource masterDataSource() {
  22. return DataSourceBuilder.create().build();
  23. }
  24. @Bean(name = "slaveDataSource")
  25. @ConfigurationProperties(prefix = "spring.datasource.slave")
  26. public DataSource slaveDataSource() {
  27. return DataSourceBuilder.create().build();
  28. }
  29. @Primary
  30. @Bean(name = "dynamicDatasource")
  31. public ClientDataSourceRouter dynamicDatasource(@Qualifier("masterDataSource") DataSource masterDataSource,
  32. @Qualifier("slaveDataSource") DataSource slaveDataSource) {
  33. ClientDataSourceRouter dataSourceRouter = new ClientDataSourceRouter();
  34. dataSourceRouter.setDefaultTargetDataSource(masterDataSource);
  35. Map<Object, Object> targetDataSources = new HashMap<>();
  36. targetDataSources.put(ClientDataSource.MASTER, masterDataSource);
  37. targetDataSources.put(ClientDataSource.SLAVE, slaveDataSource);
  38. dataSourceRouter.setTargetDataSources(targetDataSources);
  39. return dataSourceRouter;
  40. }
  41. }

3、MybatisConfig配置

  1. package com.gane.maple.jdbc.datasource;
  2. import org.apache.ibatis.session.SqlSessionFactory;
  3. import org.mybatis.spring.SqlSessionFactoryBean;
  4. import org.mybatis.spring.annotation.MapperScan;
  5. import org.springframework.beans.factory.annotation.Qualifier;
  6. import org.springframework.context.annotation.Bean;
  7. import org.springframework.context.annotation.Configuration;
  8. import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
  9. import org.springframework.jdbc.datasource.DataSourceTransactionManager;
  10. import javax.sql.DataSource;
  11. /**
  12. * @author maple
  13. * @date 2021/3/3
  14. */
  15. @Configuration
  16. @MapperScan(MybatisConfig.MAPPER_PACKAGE)
  17. public class MybatisConfig {
  18. public static final String MAPPER_PACKAGE = "com.gane.maple.dao";
  19. public static final String TYPE_ALIASES_PACKAGE = "com.gane.maple.dao.entity";
  20. public static final String MAPPER_XML_LOCATIONS = "mapper/*Mapper.xml";
  21. @Bean
  22. public SqlSessionFactory sqlSessionFactory(@Qualifier("dynamicDatasource") DataSource dataSource) throws Exception {
  23. SqlSessionFactoryBean factoryBean = new SqlSessionFactoryBean();
  24. factoryBean.setDataSource(dataSource);
  25. factoryBean.setTypeAliasesPackage(TYPE_ALIASES_PACKAGE);
  26. factoryBean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources(MAPPER_XML_LOCATIONS));
  27. return factoryBean.getObject();
  28. }
  29. @Bean
  30. public DataSourceTransactionManager transactionManager(@Qualifier("dynamicDatasource") DataSource dataSource) {
  31. return new DataSourceTransactionManager(dataSource);
  32. }
  33. }

4、创建 ClientDataSource 枚举 定义主从库

  1. package com.gane.maple.jdbc.routing;
  2. /**
  3. * @author maple
  4. * @date 2021/3/3
  5. */
  6. public enum ClientDataSource {
  7. MASTER, SLAVE
  8. }

5、创建 ClientDataSourceContextHolder 来保存 ClientDataSource

  1. package com.gane.maple.jdbc.routing;
  2. import java.util.Objects;
  3. /**
  4. * Context Holder that will hold the value for datasource routing for each different thread
  5. * (request).
  6. *
  7. * @author maple
  8. * @date 2021/3/3
  9. */
  10. public class ClientDataSourceContextHolder {
  11. private static final ThreadLocal<ClientDataSource> CONTEXT = new ThreadLocal<>();
  12. public static void set(ClientDataSource clientDataSource) {
  13. CONTEXT.set(Objects.requireNonNull(clientDataSource, "clientDatabase cannot be null"));
  14. }
  15. public static ClientDataSource getClientDatabase() {
  16. return CONTEXT.get();
  17. }
  18. public static void clear() {
  19. CONTEXT.remove();
  20. }
  21. }

6、继承 AbstractRoutingDataSource

重写 determineCurrentLookupKey 方法,返回所使用的数据源的Key(master/slave)给到 resolvedDataSources,从而通过Key从resolvedDataSources里拿到其对应的DataSource

  1. package com.gane.maple.jdbc.routing.component;
  2. import com.gane.maple.jdbc.routing.ClientDataSource;
  3. import com.gane.maple.jdbc.routing.ClientDataSourceContextHolder;
  4. import lombok.extern.slf4j.Slf4j;
  5. import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
  6. /**
  7. * {@link javax.sql.DataSource} for spring framework that will gives the desired
  8. * datasource based on the
  9. * current value stored in the {@link ClientDataSourceContextHolder}
  10. *
  11. * @author maple
  12. * @date 2021/3/3
  13. */
  14. @Slf4j
  15. public class ClientDataSourceRouter extends AbstractRoutingDataSource {
  16. @Override
  17. protected Object determineCurrentLookupKey() {
  18. ClientDataSource clientDataSource = ClientDataSourceContextHolder.getClientDatabase();
  19. if (clientDataSource == null) {
  20. log.debug("null client database, use default {}", ClientDataSource.MASTER);
  21. clientDataSource = ClientDataSource.MASTER;
  22. }
  23. log.trace("use {} as database", clientDataSource);
  24. return clientDataSource;
  25. }
  26. }

7、通过注解 DataSourceRouting 来标识走master/slave

  1. package com.gane.maple.jdbc.routing.annotation;
  2. import com.gane.maple.jdbc.routing.ClientDataSource;
  3. import java.lang.annotation.ElementType;
  4. import java.lang.annotation.Retention;
  5. import java.lang.annotation.RetentionPolicy;
  6. import java.lang.annotation.Target;
  7. /**
  8. * Indicates that a method uses a specific datasource defined in {@link ClientDataSource}.
  9. *
  10. * @author maple
  11. * @date 2021/3/3
  12. */
  13. @Target(ElementType.METHOD)
  14. @Retention(RetentionPolicy.RUNTIME)
  15. public @interface DataSourceRouting {
  16. ClientDataSource value() default ClientDataSource.MASTER;
  17. }

8、创建 DataSourceRoutingAspect,来处理注解 DataSourceRouting

  1. package com.gane.maple.jdbc.routing.component;
  2. import com.gane.maple.jdbc.routing.ClientDataSource;
  3. import com.gane.maple.jdbc.routing.ClientDataSourceContextHolder;
  4. import com.gane.maple.jdbc.routing.annotation.DataSourceRouting;
  5. import lombok.extern.slf4j.Slf4j;
  6. import org.aspectj.lang.ProceedingJoinPoint;
  7. import org.aspectj.lang.annotation.Around;
  8. import org.aspectj.lang.annotation.Aspect;
  9. import org.springframework.stereotype.Component;
  10. /**
  11. * Aspect that will mark a method to route to the desired datasource before calling the
  12. * method.
  13. *
  14. * @author maple
  15. * @date 2021/3/3
  16. */
  17. @Aspect
  18. @Component
  19. @Slf4j
  20. public class DataSourceRoutingAspect {
  21. @Around("@annotation(dataSourceRouting)")
  22. public Object aroundDataSourceRouting(ProceedingJoinPoint joinPoint, DataSourceRouting dataSourceRouting)
  23. throws Throwable {
  24. ClientDataSource previousClient = ClientDataSourceContextHolder.getClientDatabase();
  25. log.warn("Setting clientDatabase {} into DataSourceContext", dataSourceRouting.value());
  26. ClientDataSourceContextHolder.set(dataSourceRouting.value());
  27. try {
  28. return joinPoint.proceed();
  29. } finally {
  30. if (previousClient != null) {
  31. // revert context back to previous state after execute the method
  32. ClientDataSourceContextHolder.set(previousClient);
  33. } else {
  34. // there is no value being set into the context before, just clear the context
  35. // to prevent memory leak
  36. ClientDataSourceContextHolder.clear();
  37. }
  38. }
  39. }
  40. }

9、主类 ReadWriteSeparationMybatisApplication

由于我们没有使用 spring.datasource.url、spring.datasource.username 默认的配置,而是自定义的 spring.datasource.master.jdbc-url、spring.datasource.master.username 等配置,

所以我们需要排除Spring的自动配置类 DataSourceAutoConfiguration,防止在我们启动项目的时候,由于找不到  spring.datasource.url、spring.datasource.username 等配置而报了 “url” 未配置的 错误。

  1. package com.gane.maple;
  2. import org.springframework.boot.SpringApplication;
  3. import org.springframework.boot.autoconfigure.SpringBootApplication;
  4. import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
  5. @SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
  6. public class ReadWriteSeparationMybatisApplication {
  7. public static void main(String[] args) {
  8. SpringApplication.run(ReadWriteSeparationMybatisApplication.class, args);
  9. }
  10. }

10、测试

自测成功,可自行debug

TestController:

  1. package com.gane.maple.controller;
  2. import com.gane.maple.entity.User;
  3. import com.gane.maple.service.UserService;
  4. import org.springframework.beans.factory.annotation.Autowired;
  5. import org.springframework.web.bind.annotation.GetMapping;
  6. import org.springframework.web.bind.annotation.RestController;
  7. /**
  8. * @author maple
  9. * @date 2021/3/2
  10. */
  11. @RestController
  12. public class TestController {
  13. @Autowired
  14. private UserService userService;
  15. @GetMapping("/queryUser")
  16. public User queryUser() {
  17. User userFromMaster = userService.selectByUserId("1");
  18. User userFromSlave = userService.selectByUserName("maple_slave");
  19. User userFromMasterAndSlave = userService.selectFromMasterAndSlave("1", "maple_slave");
  20. User selectFromMasterAndSlaveWithDataSourceRouting = userService.selectFromMasterAndSlaveWithDataSourceRoutingInDao("1", "maple_slave");
  21. User selectFromMasterAndSlaveWithoutDataSourceRouting = userService.selectFromMasterAndSlaveWithoutDataSourceRoutingInDao("1", "maple_slave");
  22. return selectFromMasterAndSlaveWithDataSourceRouting;
  23. }
  24. }

 UserServiceImpl

  1. package com.gane.maple.service.impl;
  2. import com.gane.maple.dao.UserDAO;
  3. import com.gane.maple.entity.User;
  4. import com.gane.maple.jdbc.routing.ClientDataSource;
  5. import com.gane.maple.jdbc.routing.annotation.DataSourceRouting;
  6. import com.gane.maple.service.UserService;
  7. import org.springframework.beans.factory.annotation.Autowired;
  8. import org.springframework.stereotype.Service;
  9. /**
  10. * @Description UserServiceImpl
  11. * @Date 2020/4/24 7:41
  12. * @Created by 王弘博
  13. */
  14. @Service
  15. public class UserServiceImpl implements UserService {
  16. @Autowired
  17. private UserDAO userDAO;
  18. /**
  19. * 把 @DataSourceRouting 放进 dao 层
  20. *
  21. * @param userId
  22. * @return
  23. */
  24. @Override
  25. public User selectByUserId(String userId) {
  26. User userFromMaster = userDAO.selectByUserId(userId);
  27. System.out.println("查询master库:" + userFromMaster);
  28. return userFromMaster;
  29. }
  30. /**
  31. * 把 @DataSourceRouting 放进 dao 层
  32. *
  33. * @param userName
  34. * @return
  35. */
  36. @Override
  37. public User selectByUserName(String userName) {
  38. User userFromSlave = userDAO.selectByUserName(userName);
  39. System.out.println("查询slave库:" + userFromSlave);
  40. return userFromSlave;
  41. }
  42. /**
  43. * 把 @DataSourceRouting 放进 dao 层
  44. * 观察进入aspect几次
  45. *
  46. * @param userId
  47. * @param userName
  48. * @return
  49. */
  50. @Override
  51. public User selectFromMasterAndSlave(String userId, String userName) {
  52. User userFromMaster = userDAO.selectByUserId(userId);
  53. System.out.println("查询master库:" + userFromMaster);
  54. User userFromSlave = userDAO.selectByUserName(userName);
  55. System.out.println("查询slave库:" + userFromSlave);
  56. return userFromMaster;
  57. }
  58. /**
  59. * 把 @DataSourceRouting 放进 service 层 和 dao 层。判断具体以哪个datasource为准
  60. *
  61. * @param userId
  62. * @param userName
  63. * @return
  64. */
  65. @DataSourceRouting(value = ClientDataSource.SLAVE)
  66. @Override
  67. public User selectFromMasterAndSlaveWithDataSourceRoutingInDao(String userId, String userName) {
  68. User userFromMaster = userDAO.selectByUserId(userId);
  69. System.out.println("查询master库:" + userFromMaster);
  70. User userFromSlave = userDAO.selectByUserName(userName);
  71. System.out.println("查询slave库:" + userFromSlave);
  72. return userFromMaster;
  73. }
  74. @DataSourceRouting(value = ClientDataSource.SLAVE)
  75. @Override
  76. public User selectFromMasterAndSlaveWithoutDataSourceRoutingInDao(String userId, String userName) {
  77. User userFromMaster = userDAO.selectByUserIdWithoutDataSourceRouting(userId);
  78. System.out.println("查询master库:" + userFromMaster);
  79. User userFromSlave = userDAO.selectByUserNameWithoutDataSourceRouting(userName);
  80. System.out.println("查询slave库:" + userFromSlave);
  81. return userFromMaster;
  82. }
  83. }

UserDAO

  1. package com.gane.maple.dao;
  2. import com.gane.maple.entity.User;
  3. import com.gane.maple.jdbc.routing.ClientDataSource;
  4. import com.gane.maple.jdbc.routing.annotation.DataSourceRouting;
  5. /**
  6. * @Description UserDAO
  7. * @Date 2020/4/24 7:39
  8. * @Created by 王弘博
  9. */
  10. public interface UserDAO {
  11. @DataSourceRouting(value = ClientDataSource.MASTER)
  12. User selectByUserId(String userId);
  13. @DataSourceRouting(value = ClientDataSource.SLAVE)
  14. User selectByUserName(String userName);
  15. User selectByUserIdWithoutDataSourceRouting(String userId);
  16. User selectByUserNameWithoutDataSourceRouting(String userName);
  17. }

10、总结

  1. 我们可以把注解 DataSourceRouting 作用在 service 接口 上,也可以作用在 dao 接口 上
  2. 如果只作用在 service 接口上的话,比如配置的是 slave,那么该 service 里的所有调用 dao 的地方,都会走 slave 数据源;
  3. 如果只作用在 dao 接口上的话,比如配置的是 slave,那么该 service 里的所有调用 dao 的地方,都会走 slave 数据源;
  4. 如果 service 上配置的是 master,aDao配置的是 slave,bDao配置的是 master,当走到 service 的时候,会被 DataSourceRoutingAspect 拦截到,并赋值master给 ClientDataSourceContextHolder ,当执行到 aDao 的时候,又会被 DataSourceRoutingAspect 拦截到,拿到配置在aDao上的注解slave,重写determineCurrentLookupKey 方法里会返回 slave出去,最终走的是 slave 数据源;当执行到 bDao 的时候,又会被 DataSourceRoutingAspect 拦截到,拿到配置在bDao上的注解master,则最终执行 bDao 走的是 master 数据源;所以在dao接口上配置注解的优先级要高于在service上配置,遵循就近原则;这里需要开发人员根据自己的业务来做相应的处理。

11、源码

gitee地址:https://gitee.com/gane_maple/read-write-separation-mybatis

下篇文章我们使用 MybatisPlus 来做读写分离

【读写分离】SpringBoot整合多数据源实现读写分离(二)



所属网站分类: 技术文章 > 博客

作者:javabb

链接:http://www.javaheidong.com/blog/article/110473/9e425871b78b1ee974d8/

来源:java黑洞网

任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任

24 0
收藏该文
已收藏

评论内容:(最多支持255个字符)