数据安全之数据库字段加解密检索和前端返回脱敏？看看我这个最强解决方案
前言
数据安全一直是我们老生常谈的话题了,随着国产化的日渐推进和数字化信息改革,数据安全越来越被人们所重视。数据库作为存储、管理和检索数据的核心基础设施，其中可能包含着大量的敏感信息，如个人手机号、身份证号码、银行账户、家庭地址等信息。为了保障这些敏感信息在部分情况下被明文泄露和未授权访问等恶意行为的侵害，数据库字段敏感信息加密变得至关重要。但是数据库列一旦加密那么就牵扯到很多问题。如何对数据库字段进行加密变得非常重要,目前主要有两个解决方案:
- 数据库自带加密函数或者使用数据库自定义函数方法进行加密解密
- 使用应用代码比如java、c#等语言自带的加密解密函数库
为了助力国产化的推进下面我将用solon + easy-query对其进行实践演练和原理进行解析。
当前项目地址demo https://gitee.com/xuejm/solon-encrypt
| 方法 | 优点 | 缺点 |
|---|---|---|
| 数据库函数对 | 实现简单,占用磁盘空间少,由数据库自行实现 | 模糊搜索效率低,与数据库函数绑定,兼容性差,仅可以使用数据库提供的函数或者自行编程数据库支持的加密解密 |
| java代码 | 实现复杂,占用磁盘空间多 | 模糊搜索效率高,不与数据库函数绑定,兼容性好,可以自行扩展实现国密等对称非对称加密 |
solon
文档地址 https://xuejm.gitee.io/easy-query-doc/
GITHUB地址 https://github.com/noear/solon
GITEE地址 https://gitee.com/noear/solon
easy-qeury
文档地址 https://xuejm.gitee.io/easy-query-doc/
GITHUB地址 https://github.com/xuejmnet/easy-query
GITEE地址 https://gitee.com/xuejm/easy-query
数据库处理
这边我们以mysql为例实现数据库函数的加密解密,对数据库列进行数据保护处理。
| 方法 | 默认值 |
|---|---|
| to_base64(AES_ENCRYPT('手机号值'),'秘钥') | 将数据进行aes加密,然后进行base64编码 |
| AES_DECRYPT(from_base64('手机号列'),'秘钥') | 将数据进行base64解码,然后进行aes进行解密 |
这两个方法其实很好理解,就是通过调用数据库函数让其在数据库层面就实现了加密和解密,应用程序获取到的数据本身就是解密好了的,但是缺点就是如果需要支持列的like搜索性能会变得非常低下,因为需要对加密列进行AES_DECRYPT(from_base64('手机号列'),'秘钥')的解密处理
select id,name,AES_DECRYPT(from_base64('phone'),'秘钥') as `phone` from user where AES_DECRYPT(from_base64('phone'),'秘钥') like '%567%' 当数据量少的时候那么数据库是比较轻松并且相对性能是可以接受的,但是随着数据量的越来越大,这种sql会慢慢变成瓶颈,那么是否有一种方案可以兼顾两者呢,其实是有的.
应用代码处理
分段加密,采用分段加密可以实现like语句的模糊搜索,并且支持数据的加密,但是这种方案也有缺点就是会比较占用空间,具体原理可以看下《阿里巴巴密文字段检索方案》 https://jaq-doc.alibaba.com/docs/doc.htm?treeId=1&articleId=106213&docType=1 文章给出了具体的实现方式,约定最少4位数字或者2位中文字符4位英文字符（半角），2个中文字符（全角）,比如12345678901这么一串
分别对其进行分段[1234,2345,3456,4567,5678,6789,7890,8901]分成8份,并且对每一份进行等长数据加密,也就是加密后的结果需要等长,比如都是16位或者都是8位
| 算法/模式/填充 | 16 字节加密后数据长度 | 不满 16 字节加密后长度 | 本次采用 |
|---|---|---|---|
| AES/CBC/NoPadding | 16 | 不支持 | ❌ |
| AES/CBC/PKCS5Padding | 32 | 16 | ✅ |
| AES/CBC/ISO10126Padding | 32 | 16 | ❌ |
| AES/CFB/NoPadding | 16 | 原始数据长度 | ❌ |
| AES/CFB/PKCS5Padding | 32 | 16 | ❌ |
| AES/CFB/ISO10126Padding | 32 | 16 | ❌ |
| AES/ECB/NoPadding | 16 | 不支持 | ❌ |
| AES/ECB/PKCS5Padding | 32 | 16 | ❌ |
| AES/ECB/ISO10126Padding | 32 | 16 | ❌ |
| AES/OFB/NoPadding | 16 | 原始数据长度 | ❌ |
| AES/OFB/PKCS5Padding | 32 | 16 | ❌ |
| AES/OFB/ISO10126Padding | 32 | 16 | ❌ |
| AES/PCBC/NoPadding | 16 | 不支持 | ❌ |
| AES/PCBC/PKCS5Padding | 32 | 16 | ❌ |
| AES/PCBC/ISO10126Padding | 32 | 16 | ❌ |
加密
我们可以选择任意一种这次选择AES/CBC/PKCS5Padding 因为我们的数据原因最终肯定不满16字节,所以加密后肯定都是16字节长度,然后可以对其进行base64编码,编码后会变成24字节在对其进行合并存储
base64(aes('1234')) + base64(aes('2345')) + base64(aes('3456')) +......+ base64(aes('7890')) + base64(aes('8901'))
,然后存入数据库,之后需要对其like的话限制最小like的数据应该满足最少4位数字或者2位中文字符4位英文字符（半角），2个中文字符（全角），比如我要查询包含45678的手机号只需要现对其进行分段[4567,5678]然后对其进行加密base64(aes('4567')) + base64(aes('5678'))
解密
因为我们采用aes加密后用base64编码拼接存入数据库,所以我们只需要对数据库的数据进行获取,之后判断其长度%24是否余数为0，如果是的话那么就将其进行以每24个长度为一组进行base64解码,然后通过aes解密.解密后在对其进行拼接还原出最初的明文数据
限制或缺点
- 通过上述可以知晓解密片段必须小于16字节长度base64后的加密信息必须是定长
- 字段会扩大,原本的n位明文如果需要支持加密那么将会让字段变得非常长,但是好处是支持非常高性能的like搜索
- 建议使用到定长的数据信息中,譬如手机号,身份证号码登
- 数据库函数加密解密不支持like操作符需要注意
- 不同的数据库需要适配不同的函数
优点
- 数据库函数实现简单
- 支持任意存储对象比如es
select * from user where phone like '%xxxxx%' -- 其中xxxxx就是base64(aes('4567'))+base64(aes('5678')) 这样我们就实现了即支持加密又支持like的方式了,但是对于大部分用户来说虽然原理有了,但是实现起来还是太麻烦了,所以接下来我们就助力国产配合国产web框架和国产orm实现这两个功能,让用户在使用时无感知
实践案例
添加依赖
新建solon的web项目并且添加依赖
<dependencies> <dependency> <groupId>com.easy-query</groupId> <artifactId>sql-solon-plugin</artifactId> <version>1.3.18</version> </dependency> <dependency> <groupId>com.zaxxer</groupId> <artifactId>HikariCP</artifactId> <version>3.3.1</version> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>8.0.31</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.18.18</version> </dependency> <dependency> <groupId>org.noear</groupId> <artifactId>solon-web</artifactId> <version>2.4.2</version> </dependency> <dependency> <groupId>org.noear</groupId> <artifactId>solon.logging.simple</artifactId> <version>2.4.2</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>2.0.7</version> </dependency> </dependencies> 配置
在resources目录下新建一个app.yml文件
# 添加配置文件 db1: jdbcUrl: jdbc:mysql://127.0.0.1:3306/solon_encrypt_db?serverTimezone=GMT%2B8&characterEncoding=utf-8&useSSL=false&allowMultiQueries=true&rewriteBatchedStatements=true username: root password: root driver-class-name: com.mysql.cj.jdbc.Driver easy-query: # 配置自定义日志 # log-class: ... db1: # 支持mysql pgsql h2 mssql dameng mssql_row_number kingbase_es等其余数据库在适配中 database: mysql # 支持underlined default lower_camel_case upper_camel_case upper_underlined name-conversion: underlined default-track: true # 记录器级别的配置示例 solon.logging.logger: "root": #默认记录器配置 level: TRACE "com.zaxxer.hikari": level: WARN 具体更多参数请参考solon官方文档 和easy-query官方文档
启动类
WebApp.java
public class WebApp { public static void main(String[] args) { Solon.start(WebApp.class,args); } } 加密策略
//支持like的java方法加密解密 public class JavaEncryptionStrategy extends AbstractAesBase64EncryptionStrategy { @Override public String getIv() { return "1234567890123456"; } @Override public String getKey() { return "1234561234567890"; } } //数据库函数加密解密 public class MySQLAESColumnValueSQLConverter implements ColumnValueSQLConverter { /** * 数据加密秘钥 */ private static final String SECRET="1234567890123456"; @Override public void columnConvert(TableAvailable table, ColumnMetadata columnMetadata, SQLPropertyConverter sqlPropertyConverter, QueryRuntimeContext runtimeContext) { sqlPropertyConverter.sqlNativeSegment("AES_DECRYPT(from_base64({0}),{1})",context->{ context .expression(columnMetadata.getPropertyName())//采用变量是因为可能出现join附带别名所以需要变量 .value(SECRET) .setAlias(columnMetadata.getName()); }); } @Override public void valueConvert(TableAvailable table, ColumnMetadata columnMetadata, SQLParameter sqlParameter, SQLPropertyConverter sqlPropertyConverter, QueryRuntimeContext runtimeContext) { sqlPropertyConverter.sqlNativeSegment("to_base64(AES_ENCRYPT({0},{1}))",context->{ context.value(sqlParameter).value(SECRET); }); } } 数据库对象
//用户信息表 @Data @Table("sys_user") public class SysUser { @Column(primaryKey = true) private String id; private String name; @Column(sqlConversion = MySQLAESColumnValueSQLConverter.class) private String phone; @Encryption(strategy = JavaEncryptionStrategy.class,supportQueryLike = true) private String Address; private LocalDateTime createTime; @Navigate(value = RelationTypeEnum.OneToMany,targetProperty = "userId") private List<UserBook> books; } //为了演示复杂查询这边在新增一张用户书本表 @Data @Table("user_book") public class UserBook { @Column(primaryKey = true) private String id; private String userId; private String name; } 数据库脚本
CREATE DATABASE IF NOT EXISTS solon_encrypt_db CHARACTER SET 'utf8mb4'; create table solon_encrypt_db.sys_user ( id varchar(32) not null comment '主键ID'primary key, name varchar(50) not null comment '姓名', phone varchar(256) null comment '手机号',-- 手机号不需要模糊搜索 address varchar(512) null comment '用户地址',-- 用户地址需要模糊搜索 create_time datetime not null comment '创建时间' )comment '用户表'; create table solon_encrypt_db.user_book ( id varchar(32) not null comment '主键ID'primary key, user_id varchar(32) not null comment '姓名', name varchar(50) not null comment '姓名' )comment '用户书本表'; 配置文件
@Configuration public class DefaultConfiguration { @Bean(name = "db1",typed=true) public DataSource db1DataSource(@Inject("${db1}") HikariDataSource dataSource){ return dataSource; } @Bean public void db1QueryConfiguration(@Db("db1") QueryConfiguration configuration){ configuration.applyEncryptionStrategy(new JavaEncryptionStrategy()); configuration.applyColumnValueSQLConverter(new MySQLAESColumnValueSQLConverter()); } } 测试
新增控制器
@Controller @Mapping("/test") public class TestController { @Db private EasyQuery easyQuery; @Mapping(value = "/init",method = MethodType.GET) @Tran public String init(){ { SysUser sysUser = new SysUser(); sysUser.setId("1"); sysUser.setName("用户1"); sysUser.setPhone("12345678901"); sysUser.setAddress("浙江省绍兴市越城区城市广场1234号"); sysUser.setCreateTime(LocalDateTime.now()); ArrayList<UserBook> userBooks = new ArrayList<>(); UserBook userBook = new UserBook(); userBook.setId("1"); userBook.setUserId("1"); userBook.setName("语文"); userBooks.add(userBook); UserBook userBook1 = new UserBook(); userBook1.setId("2"); userBook1.setUserId("1"); userBook1.setName("数学"); userBooks.add(userBook1); easyQuery.insertable(sysUser).executeRows(); easyQuery.insertable(userBooks).executeRows(); } { SysUser sysUser = new SysUser(); sysUser.setId("2"); sysUser.setName("用户2"); sysUser.setPhone("19012345678"); sysUser.setAddress("浙江省杭州市上城区武林广场1234号"); sysUser.setCreateTime(LocalDateTime.now()); ArrayList<UserBook> userBooks = new ArrayList<>(); UserBook userBook = new UserBook(); userBook.setId("3"); userBook.setUserId("2"); userBook.setName("语文"); userBooks.add(userBook); UserBook userBook1 = new UserBook(); userBook1.setId("4"); userBook1.setUserId("2"); userBook1.setName("英语"); userBooks.add(userBook1); easyQuery.insertable(sysUser).executeRows(); easyQuery.insertable(userBooks).executeRows(); } return "初始化完成"; } } ==> Preparing: INSERT INTO `sys_user` (`id`,`name`,`phone`,`address`,`create_time`) VALUES (?,?,to_base64(AES_ENCRYPT(?,?)),?,?) ==> Parameters: 1(String),用户1(String),12345678901(String),1234567890123456(String),miaKEctf5bGBi4yFHvSV6A==i9CdpEU+Ji+g0pPYOpTcWA==9RprkhoOPwcA13Ye0eE0NA==f0ryEfO7ajP2qQ9Yia/dwA==bFZZS42+JmMlvK+6t9a2xQ==O+TkblfoJWgGu6o/w3RuBQ==urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==8u/RP9cyz8l7udgay5Tbnw==oLi10kERsXzxuJdSFAZN9w==Sgm9i3O/7FtvC4ryFziNug==9gkm5m1HD8qS4ITJ0r/W4A==zppH8USinNqLsEPxJ2jfiQ==RY3Ji2Exl1StrrdrzSVvDQ==lMnY0leaGzXqeK/mukEIQQ==NlthvCsk4jaQkEioF/SWsA==(String),2023-08-13T09:17:01.503(LocalDateTime) <== Total: 1 ==> Preparing: INSERT INTO `user_book` (`id`,`user_id`,`name`) VALUES (?,?,?) ==> Parameters: 1(String),1(String),语文(String) <== Total: 1 ==> Preparing: INSERT INTO `user_book` (`id`,`user_id`,`name`) VALUES (?,?,?) ==> Parameters: 2(String),1(String),数学(String) <== Total: 1 ==> Preparing: INSERT INTO `sys_user` (`id`,`name`,`phone`,`address`,`create_time`) VALUES (?,?,to_base64(AES_ENCRYPT(?,?)),?,?) ==> Parameters: 2(String),用户2(String),19012345678(String),1234567890123456(String),miaKEctf5bGBi4yFHvSV6A==i9CdpEU+Ji+g0pPYOpTcWA==JdzWF3gRqqCuHO+fiRTsGQ==Ydc2v/Ghy3MbHTvTiLqHIg==B9zPkalGKbJMzyFgw8W6bA==yIJYfG5BGqQnnR5+GhdV4g==V7Zu1p3qHPjOBj+vAc1MQA==+n2a0u3gq1V4L8aKa/eyEg==MEsrlm3QnRdt4entjjf97w==rBJCNrGBSjKI6T77OXD2dg==k75blBdYdH81FSIB4AVjeA==9gkm5m1HD8qS4ITJ0r/W4A==zppH8USinNqLsEPxJ2jfiQ==RY3Ji2Exl1StrrdrzSVvDQ==lMnY0leaGzXqeK/mukEIQQ==NlthvCsk4jaQkEioF/SWsA==(String),2023-08-13T09:17:01.775(LocalDateTime) <== Total: 1 ==> Preparing: INSERT INTO `user_book` (`id`,`user_id`,`name`) VALUES (?,?,?) ==> Parameters: 3(String),2(String),语文(String) <== Total: 1 ==> Preparing: INSERT INTO `user_book` (`id`,`user_id`,`name`) VALUES (?,?,?) ==> Parameters: 4(String),2(String),英语(String) <== Total: 1 启动程序访问http://localhost:8080/test/init,插入对应的初始化数据
查询数据
@Mapping(value = "/query",method = MethodType.GET) public Object query(){ List<SysUser> list = easyQuery.queryable(SysUser.class) .include(o -> o.many(SysUser::getBooks)) .toList(); return list; } [{"id":"1","name":"用户1","phone":"12345678901","Address":"浙江省绍兴市越城区城市广场1234号","createTime":1691889422000,"books":[{"id":"1","userId":"1","name":"语文"},{"id":"2","userId":"1","name":"数学"}]},{"id":"2","name":"用户2","phone":"19012345678","Address":"浙江省杭州市上城区武林广场1234号","createTime":1691889422000,"books":[{"id":"3","userId":"2","name":"语文"},{"id":"4","userId":"2","name":"英语"}]}] 按用户加密地址模糊匹配
@Mapping(value = "/queryByAddress",method = MethodType.GET) public Object queryByAddress(){ SysUser user = easyQuery.queryable(SysUser.class) .where(o->o.like(SysUser::getAddress,"越城区")) .firstOrNull(); return user; } {"id":"1","name":"用户1","phone":"12345678901","Address":"浙江省绍兴市越城区城市广场1234号","createTime":1691889422000} ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `address` LIKE ? LIMIT 1 ==> Parameters: 1234567890123456(String),%urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==%(String) <== Time Elapsed: 23(ms) <== Total: 1 分析
浙江省绍兴市越城区城市广场1234号:miaKEctf5bGBi4yFHvSV6A==i9CdpEU+Ji+g0pPYOpTcWA==9RprkhoOPwcA13Ye0eE0NA==f0ryEfO7ajP2qQ9Yia/dwA==bFZZS42+JmMlvK+6t9a2xQ==O+TkblfoJWgGu6o/w3RuBQ==**urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==**8u/RP9cyz8l7udgay5Tbnw==oLi10kERsXzxuJdSFAZN9w==Sgm9i3O/7FtvC4ryFziNug==9gkm5m1HD8qS4ITJ0r/W4A==zppH8USinNqLsEPxJ2jfiQ==RY3Ji2Exl1StrrdrzSVvDQ==lMnY0leaGzXqeK/mukEIQQ==NlthvCsk4jaQkEioF/SWsA==
越城区加密后:urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==
查询用户书本表并且获取用户书本对应的用户名
@Data public class UserBookVO { private String id; private String userId; private String name; private String userName; private String userPhone; /** * 返回vo结果如果没有该注解将是返回数据库对应的列的原始数据 */ @Encryption(strategy = JavaEncryptionStrategy.class) private String userAddress; } @Mapping(value = "/queryBook",method = MethodType.GET) public Object queryBook(){ //无需考虑表别名所属`t`还是`t1`等 List<UserBookVO> userBooks = easyQuery.queryable(UserBook.class) .leftJoin(SysUser.class, (t, t1) -> t.eq(t1, UserBook::getUserId, SysUser::getId)) .where((t, t1) -> t1.like(SysUser::getAddress, "越城区")) .select(UserBookVO.class, (t, t1) -> t.columnAll() .then(t1) .columnAs(SysUser::getName, UserBookVO::getUserName) .columnAs(SysUser::getPhone, UserBookVO::getUserPhone) .columnAs(SysUser::getAddress, UserBookVO::getUserAddress) ) .toList(); return userBooks; } [{"id":"1","userId":"1","name":"语文","userName":"用户1","userPhone":"12345678901","userAddress":"浙江省绍兴市越城区城市广场1234号"},{"id":"2","userId":"1","name":"数学","userName":"用户1","userPhone":"12345678901","userAddress":"浙江省绍兴市越城区城市广场1234号"}] ==> Preparing: SELECT t.`id`,t.`user_id`,t.`name`,t1.`name` AS `user_name`,AES_DECRYPT(from_base64(t1.`phone`),?) AS `user_phone`,t1.`address` AS `user_address` FROM `user_book` t LEFT JOIN `sys_user` t1 ON t.`user_id` = t1.`id` WHERE t1.`address` LIKE ? ==> Parameters: 1234567890123456(String),%urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==%(String) <== Time Elapsed: 21(ms) <== Total: 2 修改手机号和地址
@Mapping(value = "/updateFull",method = MethodType.GET) public Object updateFull(){ SysUser user = easyQuery.queryable(SysUser.class).whereById("1").firstOrNull(); user.setPhone("12312312312"); user.setAddress("浙江省杭州市上城区0987号"); easyQuery.updatable(user).executeRows(); return null; } ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `id` = ? LIMIT 1 ==> Parameters: 1234567890123456(String),1(String) <== Time Elapsed: 35(ms) <== Total: 1 ==> Preparing: UPDATE `sys_user` SET `name` = ?,`phone` = to_base64(AES_ENCRYPT(?,?)),`address` = ?,`create_time` = ? WHERE `id` = ? ==> Parameters: 用户1(String),12312312312(String),1234567890123456(String),miaKEctf5bGBi4yFHvSV6A==i9CdpEU+Ji+g0pPYOpTcWA==JdzWF3gRqqCuHO+fiRTsGQ==Ydc2v/Ghy3MbHTvTiLqHIg==B9zPkalGKbJMzyFgw8W6bA==yIJYfG5BGqQnnR5+GhdV4g==V7Zu1p3qHPjOBj+vAc1MQA==+n2a0u3gq1V4L8aKa/eyEg==+TZVS8QvlcDWZ4UTb8nhYQ==8ovX18Yz7LBizRX/aUCc+w==UY1+e/bka/X9nUuHla9C/w==VUW3oU+N4hm3mVZFJeal7Q==(String),2023-08-13T09:17:02(LocalDateTime),1(String) <== Total: 1 我只改了两个字段但是sql生成了全字段,下面就进行优化
@Mapping(value = "/updateTrack",method = MethodType.GET) @EasyQueryTrack public Object updateTrack(){ SysUser user = easyQuery.queryable(SysUser.class).whereById("1").firstOrNull(); user.setPhone("123123123123"); user.setAddress("绍兴市越城区87651号"); easyQuery.updatable(user).executeRows(); return null; } ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `id` = ? LIMIT 1 ==> Parameters: 1234567890123456(String),1(String) <== Time Elapsed: 10(ms) <== Total: 1 ==> Preparing: UPDATE `sys_user` SET `phone` = to_base64(AES_ENCRYPT(?,?)),`address` = ? WHERE `id` = ? ==> Parameters: 123123123123(String),1234567890123456(String),f0ryEfO7ajP2qQ9Yia/dwA==bFZZS42+JmMlvK+6t9a2xQ==O+TkblfoJWgGu6o/w3RuBQ==urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==ClirMJiuQjNV88CrpX4VKw==HWyz2JynN1dsAf5+bICC1A==X85Lef9VTnHIYUq4iTK4UQ==L32UeQ8jeCATh718YrxeQg==nozQIvfNzmvfkNQL2bI7RQ==(String),1(String) <== Total: 1 直接更新
@Mapping(value = "/updateSet",method = MethodType.GET) public Object updateSet(){ easyQuery.updatable(SysUser.class) .set(SysUser::getPhone,"1234567") .set(SysUser::getAddress,"绍兴市越城区123号") .whereById("1") .executeRows(); return null; } ==> Preparing: UPDATE `sys_user` SET `phone` = to_base64(AES_ENCRYPT(?,?)),`address` = ? WHERE `id` = ? ==> Parameters: 1234567(String),1234567890123456(String),f0ryEfO7ajP2qQ9Yia/dwA==bFZZS42+JmMlvK+6t9a2xQ==O+TkblfoJWgGu6o/w3RuBQ==urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==xznBF4y5j5N4cik+fWz0jw==I3FW5ECJDfcM6yNWsYlH/g==6QBhErf/Lxjd5my5p3v4wg==(String),1(String) <== Total: 1 数据库加密函数手机号检索
@Mapping(value = "/queryByPhone",method = MethodType.GET) public Object queryByPhone(){ SysUser sysUser = easyQuery.queryable(SysUser.class) .where(o -> o.eq(SysUser::getPhone,"1234567")) .firstOrNull(); return sysUser; } {"id":"1","name":"用户1","phone":"1234567","Address":"绍兴市越城区123号","createTime":1691889422000} //默认只支持全匹配查询,因为解密函数在数据库列上是十分低效的 ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `phone` = to_base64(AES_ENCRYPT(?,?)) LIMIT 1 ==> Parameters: 1234567890123456(String),1234567(String),1234567890123456(String) <== Time Elapsed: 27(ms) <== Total: 1 如果你希望数据库加密列也可以被检索可以自定义数据库片段函数来实现
@Mapping(value = "/queryPhone",method = MethodType.GET) public Object queryPhone(){ SysUser sysUser = easyQuery.queryable(SysUser.class) .where(o -> o.sqlNativeSegment("AES_DECRYPT(from_base64({0}),{1}) like {2}",context->{ context.expression(SysUser::getPhone)//列 .value("1234567890123456")//秘钥 .value("%123%");//like值 })) .firstOrNull(); return sysUser; } {"id":"1","name":"用户1","phone":"1234567","Address":"绍兴市越城区123号","createTime":1691889422000} ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE AES_DECRYPT(from_base64(`phone`),?) like ? LIMIT 1 ==> Parameters: 1234567890123456(String),1234567890123456(String),%123%(String) <== Time Elapsed: 11(ms) <== Total: 1 数据脱敏
很多时候我们的数据库对象不需要脱敏数据,而是我们的VO对象在返回的时候需要对其数据进行脱敏,所以我们可以通过ValueConverter来实现
public class AddressMask implements ValueConverter<String,String> { @Override public String serialize(String s) { //因为是VO对象不需要实现 throw new UnsupportedOperationException(); } @Override public String deserialize(Class<String> propertyClass, String value) { if(EasyStringUtil.isBlank(value)){ return value; } if(value.length()>5){ return value.substring(0,3)+"***"; } return value; } } @Data public class UserBookVO { private String id; private String userId; private String name; private String userName; private String userPhone; /** * 返回vo结果如果没有该注解将是返回数据库对应的列的原始数据 */ @Encryption(strategy = JavaEncryptionStrategy.class) @Column(conversion = AddressMask.class)//添加地址脱敏 private String userAddress; } @Configuration public class DefaultConfiguration { @Bean(name = "db1",typed=true) public DataSource db1DataSource(@Inject("${db1}") HikariDataSource dataSource){ return dataSource; } @Bean public void db1QueryConfiguration(@Db("db1") QueryConfiguration configuration){ configuration.applyEncryptionStrategy(new JavaEncryptionStrategy()); configuration.applyEncryptionStrategy(new JavaSM4EncryptionStrategy()); configuration.applyColumnValueSQLConverter(new MySQLAESColumnValueSQLConverter()); configuration.applyValueConverter(new AddressMask()); } } @Mapping(value = "/queryBookMask",method = MethodType.GET) public Object queryBookMask(){ List<UserBookVO> userBooks = easyQuery.queryable(UserBook.class) .leftJoin(SysUser.class, (t, t1) -> t.eq(t1, UserBook::getUserId, SysUser::getId)) .where((t, t1) -> t1.like(SysUser::getAddress, "越城区")) .select(UserBookVO.class, (t, t1) -> t.columnAll() .then(t1) .columnAs(SysUser::getName, UserBookVO::getUserName) .columnAs(SysUser::getPhone, UserBookVO::getUserPhone) .columnAs(SysUser::getAddress, UserBookVO::getUserAddress) ) .toList(); return userBooks; } //数据地址被解密后正确的被脱敏返回到前端 [{"id":"1","userId":"1","name":"语文","userName":"用户1","userPhone":"1234567","userAddress":"绍兴市***"},{"id":"2","userId":"1","name":"数学","userName":"用户1","userPhone":"1234567","userAddress":"绍兴市***"}] ==> Preparing: SELECT t.`id`,t.`user_id`,t.`name`,t1.`name` AS `user_name`,AES_DECRYPT(from_base64(t1.`phone`),?) AS `user_phone`,t1.`address` AS `user_address` FROM `user_book` t LEFT JOIN `sys_user` t1 ON t.`user_id` = t1.`id` WHERE t1.`address` LIKE ? ==> Parameters: 1234567890123456(String),%urDZztVNP45UWWQrQsneOg==+n2a0u3gq1V4L8aKa/eyEg==%(String) <== Time Elapsed: 13(ms) <== Total: 2 SM4
方便起见我们这边直接引入bouncycastle包
<dependency> <groupId>org.bouncycastle</groupId> <artifactId>bcprov-jdk15to18</artifactId> <version>1.64</version> </dependency> 编写sm4静态工具类
public class SM4Util { static{ Security.addProvider(new BouncyCastleProvider()); } //算法名称 public static final String ALGORITHM_NAME = "SM4"; //ECB P5填充 public static final String ALGORITHM_NAME_ECB_PADDING = "SM4/ECB/PKCS5Padding"; //CBC P5填充 public static final String ALGORITHM_NAME_CBC_PADDING = "SM4/CBC/PKCS5Padding"; //密钥长度 public static final int DEFAULT_KEY_SIZE = 128; /** * 获取密钥 * @return byte * @throws NoSuchAlgorithmException * @throws NoSuchProviderException */ public static byte[] generateKey() throws NoSuchAlgorithmException, NoSuchProviderException { return generateKey(DEFAULT_KEY_SIZE); } /** * 获取指定长度密钥 * @param keySize 密钥的长度 * @return byte * @throws NoSuchAlgorithmException * @throws NoSuchProviderException */ public static byte[] generateKey(int keySize) throws NoSuchAlgorithmException, NoSuchProviderException { KeyGenerator kg = KeyGenerator.getInstance(ALGORITHM_NAME, BouncyCastleProvider.PROVIDER_NAME); kg.init(keySize, new SecureRandom()); return kg.generateKey().getEncoded(); } public static String encrypt(String plaintext, String key, Charset charset) { try { byte[] encrypt = encrypt_Ecb_Padding(plaintext.getBytes(charset), key.getBytes(charset)); return new String(EasyBase64Util.encode(encrypt),charset); } catch (Exception e) { throw new RuntimeException(e); } } /** * ECB P5填充加密 * @param key 密钥 * @param data 明文数据 * @return byte * @throws InvalidKeyException * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException * @throws IllegalBlockSizeException * @throws BadPaddingException */ public static byte[] encrypt_Ecb_Padding(byte[] data,byte[] key) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException, IllegalBlockSizeException, BadPaddingException { Cipher cipher = generateEcbCipher(ALGORITHM_NAME_ECB_PADDING, Cipher.ENCRYPT_MODE, key); return cipher.doFinal(data); } public static String decrypt(String content, String key, Charset charset) { try { byte[] decrypt = decrypt_Ecb_Padding(EasyBase64Util.decode(content.getBytes(charset)), key.getBytes(charset)); return new String(decrypt,charset); } catch (Exception e) { throw new RuntimeException(e); } } /** * ECB P5填充解密 * @param key 密钥 * @param cipherText 加密后的数据 * @return byte * @throws IllegalBlockSizeException * @throws BadPaddingException * @throws InvalidKeyException * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException */ public static byte[] decrypt_Ecb_Padding( byte[] cipherText,byte[] key) throws IllegalBlockSizeException, BadPaddingException, InvalidKeyException, NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException { Cipher cipher = generateEcbCipher(ALGORITHM_NAME_ECB_PADDING, Cipher.DECRYPT_MODE, key); return cipher.doFinal(cipherText); } /** * CBC P5填充加密 * @param key 密钥 * @param iv 偏移量 * @param data 明文数据 * @return byte * @throws InvalidKeyException * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException * @throws IllegalBlockSizeException * @throws BadPaddingException * @throws InvalidAlgorithmParameterException */ public static byte[] encrypt_Cbc_Padding(byte[] key, byte[] iv, byte[] data) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException, IllegalBlockSizeException, BadPaddingException, InvalidAlgorithmParameterException { Cipher cipher = generateCbcCipher(ALGORITHM_NAME_CBC_PADDING, Cipher.ENCRYPT_MODE, key, iv); return cipher.doFinal(data); } /** * CBC P5填充解密 * @param key 密钥 * @param iv 偏移量 * @param cipherText 加密数据 * @return byte * @throws IllegalBlockSizeException * @throws BadPaddingException * @throws InvalidKeyException * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException * @throws InvalidAlgorithmParameterException */ public static byte[] decrypt_Cbc_Padding(byte[] key, byte[] iv, byte[] cipherText) throws IllegalBlockSizeException, BadPaddingException, InvalidKeyException, NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException, InvalidAlgorithmParameterException { Cipher cipher = generateCbcCipher(ALGORITHM_NAME_CBC_PADDING, Cipher.DECRYPT_MODE, key, iv); return cipher.doFinal(cipherText); } /** * ECB P5填充加解密Cipher初始化 * @param algorithmName 算法名称 * @param mode 1 加密 2解密 * @param key 密钥 * @return Cipher * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException * @throws InvalidKeyException */ private static Cipher generateEcbCipher(String algorithmName, int mode, byte[] key) throws NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException, InvalidKeyException { Cipher cipher = Cipher.getInstance(algorithmName, BouncyCastleProvider.PROVIDER_NAME); Key sm4Key = new SecretKeySpec(key, ALGORITHM_NAME); cipher.init(mode, sm4Key); return cipher; } /** * CBC P5填充加解密Cipher初始化 * @param algorithmName 算法名称 * @param mode 1 加密 2解密 * @param key 密钥 * @param iv 偏移量 * @return Cipher * @throws InvalidKeyException * @throws InvalidAlgorithmParameterException * @throws NoSuchAlgorithmException * @throws NoSuchProviderException * @throws NoSuchPaddingException */ private static Cipher generateCbcCipher(String algorithmName, int mode, byte[] key, byte[] iv) throws InvalidKeyException, InvalidAlgorithmParameterException, NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException { Cipher cipher = Cipher.getInstance(algorithmName, BouncyCastleProvider.PROVIDER_NAME); Key sm4Key = new SecretKeySpec(key, ALGORITHM_NAME); IvParameterSpec ivParameterSpec = new IvParameterSpec(iv); cipher.init(mode, sm4Key, ivParameterSpec); return cipher; } } 编写加密策略
public class JavaSM4EncryptionStrategy implements EncryptionStrategy { private static final Log log = LogFactory.getLog(JavaSM4EncryptionStrategy.class); private static final String key="1234567890123456"; @Override public Object encrypt(Class<?> entityClass, String propertyName, Object plaintext) { if (plaintext == null) { return null; } try { return doEncrypt(plaintext); }catch (Exception exception){ log.error(EasyClassUtil.getInstanceSimpleName(this)+" "+ EasyClassUtil.getSimpleName(entityClass)+"."+ "."+propertyName+" decrypt error:" + plaintext, exception); throw exception; } } protected Object doEncrypt(Object plaintext){ String plaintextString = plaintext.toString(); //4表示多少单位长度为一组,1表示非中文的情况下1个字符为一个单位长度,2表示一个中文字符为2个单位长度 List<String> stringCharSegments = EasyStringUtil.getStringCharSegments(plaintextString, 4,1,2); //符合要求譬如最少4个非中文字符或者2个中文字的情况下,可以选择抛错重写或者直接加密对应的值 if (EasyCollectionUtil.isEmpty(stringCharSegments)) { stringCharSegments.add(plaintextString); } StringBuilder stringBuilder = new StringBuilder(); for (String stringCharSegment : stringCharSegments) { String str= SM4Util.encrypt(stringCharSegment,key, StandardCharsets.UTF_8); stringBuilder.append(str); } return stringBuilder.toString(); } @Override public Object decrypt(Class<?> entityClass,String propertyName,Object ciphertext) { try { return doDecrypt(entityClass,propertyName,ciphertext); } catch (Exception exception) { log.error(EasyClassUtil.getInstanceSimpleName(this)+" "+ EasyClassUtil.getSimpleName(entityClass)+"."+ "."+propertyName+" decrypt error:" + ciphertext, exception); throw exception; } } private Object doDecrypt(Class<?> entityClass,String propertyName,Object ciphertext) { if (ciphertext == null) { return null; } String ciphertextString = ciphertext.toString(); if (ciphertextString.length() % 24 != 0) { throw new IllegalArgumentException(EasyClassUtil.getSimpleName(entityClass)+"."+propertyName+" decrypt cant decode base64:" + ciphertext); } List<String> segments = EasyStringUtil.splitString(ciphertextString, 24); StringBuilder stringBuilder = new StringBuilder(); for (int i = 0; i < segments.size(); i++) { String str=SM4Util.decrypt(segments.get(i),key,StandardCharsets.UTF_8); boolean last = i == (segments.size() - 1); if (last) { stringBuilder.append(str); } else { stringBuilder.append(str.charAt(0)); } } return stringBuilder.toString(); } } @Configuration public class DefaultConfiguration { @Bean(name = "db1",typed=true) public DataSource db1DataSource(@Inject("${db1}") HikariDataSource dataSource){ return dataSource; } @Bean public void db1QueryConfiguration(@Db("db1") QueryConfiguration configuration){ configuration.applyEncryptionStrategy(new JavaEncryptionStrategy()); configuration.applyEncryptionStrategy(new JavaSM4EncryptionStrategy());//添加sm4 configuration.applyColumnValueSQLConverter(new MySQLAESColumnValueSQLConverter()); } } @Data @Table("sys_user") public class SysUserSM4 { @Column(primaryKey = true) private String id; private String name; @Column(sqlConversion = MySQLAESColumnValueSQLConverter.class) private String phone; @Encryption(strategy = JavaSM4EncryptionStrategy.class,supportQueryLike = true) private String Address; private LocalDateTime createTime; } @Mapping(value = "/initSM4",method = MethodType.GET) @Tran public String initSM4(){ { SysUserSM4 sysUser = new SysUserSM4(); sysUser.setId("5"); sysUser.setName("用户5"); sysUser.setPhone("12345678901"); sysUser.setAddress("浙江省绍兴市越城区城市广场1234号"); sysUser.setCreateTime(LocalDateTime.now()); easyQuery.insertable(sysUser).executeRows(); } { SysUserSM4 sysUser = new SysUserSM4(); sysUser.setId("6"); sysUser.setName("用户6"); sysUser.setPhone("19012345678"); sysUser.setAddress("浙江省杭州市上城区武林广场1234号"); sysUser.setCreateTime(LocalDateTime.now()); easyQuery.insertable(sysUser).executeRows(); } return "初始化完成"; } ==> Preparing: INSERT INTO `sys_user` (`id`,`name`,`phone`,`address`,`create_time`) VALUES (?,?,to_base64(AES_ENCRYPT(?,?)),?,?) ==> Parameters: 5(String),用户5(String),12345678901(String),1234567890123456(String),KHxVEDHBxB0x9kgAltKrMA==llZIL8h9i+2b7sPaSt6qpw==/WFPdFPf569dkeGI2Q9r9A==CAvnuJp9Lz30LTVaZi5U5A==JKhjq5f94+MJgJK7Fc4lRA==flZDUtkyeOZJrdUE0DxlZg==jlVLlk9iVJCOCdln+G11Mg==wFIL7wK7nBctC0slOEomrg==zztIbbTcuUyyS+Zj2JgQ1w==X6DWRoQjqCunrA9w6ZlJ3Q==hN0Bm2/qS3XRK2Xxe8/MIw==iDZGTAU/WlMkwLAoiYuh8Q==R4pbp78Ig7qHCLzn9IF7rw==woPrxebr8Xvyo1qG8QxAUA==65pvnL+1Og20OW+xunqHCA==vJKXxvzbvWtZB9hrWrioCg==(String),2023-08-13T22:28:27.204(LocalDateTime) <== Total: 1 ==> Preparing: INSERT INTO `sys_user` (`id`,`name`,`phone`,`address`,`create_time`) VALUES (?,?,to_base64(AES_ENCRYPT(?,?)),?,?) ==> Parameters: 6(String),用户6(String),19012345678(String),1234567890123456(String),KHxVEDHBxB0x9kgAltKrMA==llZIL8h9i+2b7sPaSt6qpw==66gmuLlsoaX1sHDabqd/XA==YGGmh56Hc5MS+Wf8dZdl8w==SjvWmsqOacq5Kui8xDCxxw==cQrhVkPp3Hf5s/GKHpNOaw==BDvJOrbklVQGHodEa+eyCA==wFIL7wK7nBctC0slOEomrg==BZcCFFYJzjQzZ7R23fmOUA==M8WFvyffOu6BeTpQgghhUA==Jw8BjPktNN8CPRyi1f5Vrg==iDZGTAU/WlMkwLAoiYuh8Q==R4pbp78Ig7qHCLzn9IF7rw==woPrxebr8Xvyo1qG8QxAUA==65pvnL+1Og20OW+xunqHCA==vJKXxvzbvWtZB9hrWrioCg==(String),2023-08-13T22:28:27.622(LocalDateTime) <== Total: 1 查询对应的数据
@Mapping(value = "/querySM4",method = MethodType.GET) public Object querySM4(){ List<SysUserSM4> list = easyQuery.queryable(SysUserSM4.class) .whereByIds(Arrays.asList("5", "6")) .toList(); return list; } [{"id":"5","name":"用户5","phone":"12345678901","Address":"浙江省绍兴市越城区城市广场1234号","createTime":1691936907000},{"id":"6","name":"用户6","phone":"19012345678","Address":"浙江省杭州市上城区武林广场1234号","createTime":1691936908000}] ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `id` IN (?,?) ==> Parameters: 1234567890123456(String),5(String),6(String) <== Time Elapsed: 14(ms) <== Total: 2 模糊查询
@Mapping(value = "/queryLikeSM4",method = MethodType.GET) public Object queryLikeSM4(){ List<SysUserSM4> list = easyQuery.queryable(SysUserSM4.class) .whereByIds(Arrays.asList("5", "6")) .where(o->o.like(SysUserSM4::getAddress,"武林广场")) .toList(); return list; } [{"id":"6","name":"用户6","phone":"19012345678","Address":"浙江省杭州市上城区武林广场1234号","createTime":1691936908000}] ==> Preparing: SELECT `id`,`name`,AES_DECRYPT(from_base64(`phone`),?) AS `phone`,`address`,`create_time` FROM `sys_user` WHERE `id` IN (?,?) AND `address` LIKE ? ==> Parameters: 1234567890123456(String),5(String),6(String),%M8WFvyffOu6BeTpQgghhUA==Jw8BjPktNN8CPRyi1f5Vrg==iDZGTAU/WlMkwLAoiYuh8Q==%(String) <== Time Elapsed: 38(ms) <== Total: 1 注意例子里面采用的是简单的ecb模式,您可以自定义或者使用cbc,cbc模式需要多一个iv向量
支持我们实现了数据库列的加密,数据库函数对和java代码自定义,其中java代码支持like,数据库需要支持like就会消耗大量的计算,需要对所有加密数据在数据库层面进行解密后才可以
最后
看到这边您应该已经知道了solon国产框架的简洁和easy-query的便捷,如果本篇文章对您有帮助或者您觉得还行请给我一个星星表示支持谢谢
当前项目地址demo https://gitee.com/xuejm/solon-encrypt
easy-qeury
文档地址 https://xuejm.gitee.io/easy-query-doc/
GITHUB地址 https://github.com/xuejmnet/easy-query
GITEE地址 https://gitee.com/xuejm/easy-query
solon
文档地址 https://xuejm.gitee.io/easy-query-doc/
