Redis穿透问题解决方案详解编程语言

缓存穿透

 缓存穿透是指用户查询数据,在数据库没有,自然在缓存中也不会有。这样就导致用户查询的时候,在缓存中找不到,每次都要去数据库再查询一遍,然后返回空。这样请求就绕过缓存直接查数据库,这也是经常提的缓存命中率问题。

解决的办法就是:如果查询数据库也为空,直接设置一个默认值存放到缓存,这样第二次到缓冲中获取就有值了,而不会继续访问数据库,这种办法最简单粗暴。

 

把空结果,也给缓存起来,这样下次同样的请求就可以直接返回空了,即可以避免当查询的值为空时引起的缓存穿透。同时也可以单独设置个缓存区域存储空值,对要查询的key进行预先校验,然后再放行给后面的正常缓存处理逻辑。

 

查询查不到的数据,在缓存中没有,而直接走了数据库! 反反复复的去这么做就崩溃了哦

Redis穿透问题解决方案详解编程语言

4没有,redis中没有,然后去DB查询,会导致雪崩效应。称之为 穿透效应。

 

穿透 产生的原因:客户端随机生成不同的key,在redis缓存中没有该数据,数据库也没有该数据。这样的话可能导致一直发生jdbc连接

 

解决方案:

   1、通过网关判断客户端传入对应key的规则,不符合数据库查询规则,直接返回空 

   2、如果使用的key数据库查询不到的话,直接在redis中存一份null结果。 

      在存入id为4的数据库的时候,直接清除对应redis为4的缓存(此时是空哈)

 

废话不多说,上代码:

 pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
  <modelVersion>4.0.0</modelVersion> 
  <groupId>com.toov5.architect</groupId> 
  <artifactId>architect</artifactId> 
  <version>0.0.1-SNAPSHOT</version> 
  <parent> 
		<groupId>org.springframework.boot</groupId> 
		<artifactId>spring-boot-starter-parent</artifactId> 
		<version>2.0.0.RELEASE</version> 
	</parent> 
	<dependencies> 
		<!-- SpringBoot 对lombok 支持 --> 
		<dependency> 
			<groupId>org.projectlombok</groupId> 
			<artifactId>lombok</artifactId> 
		</dependency> 
		<!-- SpringBoot web 核心组件 --> 
		<dependency> 
			<groupId>org.springframework.boot</groupId> 
			<artifactId>spring-boot-starter-web</artifactId> 
		</dependency> 
		<dependency> 
			<groupId>org.springframework.boot</groupId> 
			<artifactId>spring-boot-starter-tomcat</artifactId> 
		</dependency> 
		<!-- SpringBoot 外部tomcat支持 --> 
		<dependency> 
			<groupId>org.apache.tomcat.embed</groupId> 
			<artifactId>tomcat-embed-jasper</artifactId> 
		</dependency> 
		<!-- springboot-log4j --> 
		<dependency> 
			<groupId>org.springframework.boot</groupId> 
			<artifactId>spring-boot-starter-log4j</artifactId> 
			<version>1.3.8.RELEASE</version> 
		</dependency> 
		<!-- springboot-aop 技术 --> 
		<dependency> 
			<groupId>org.springframework.boot</groupId> 
			<artifactId>spring-boot-starter-aop</artifactId> 
		</dependency> 
		<!-- https://mvnrepository.com/artifact/commons-lang/commons-lang --> 
		<dependency> 
			<groupId>commons-lang</groupId> 
			<artifactId>commons-lang</artifactId> 
			<version>2.6</version> 
		</dependency> 
		<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient --> 
		<dependency> 
			<groupId>org.apache.httpcomponents</groupId> 
			<artifactId>httpclient</artifactId> 
		</dependency> 
		<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson --> 
		<dependency> 
			<groupId>com.alibaba</groupId> 
			<artifactId>fastjson</artifactId> 
			<version>1.2.47</version> 
		</dependency> 
		<dependency> 
			<groupId>javax.servlet</groupId> 
			<artifactId>jstl</artifactId> 
		</dependency> 
		<dependency> 
			<groupId>taglibs</groupId> 
			<artifactId>standard</artifactId> 
			<version>1.1.2</version> 
		</dependency> 
		<!--开启 cache 缓存 --> 
		<dependency> 
			<groupId>org.springframework.boot</groupId> 
			<artifactId>spring-boot-starter-cache</artifactId> 
		</dependency> 
		<!-- ehcache缓存 --> 
		<dependency> 
			<groupId>net.sf.ehcache</groupId> 
			<artifactId>ehcache</artifactId> 
			<version>2.9.1</version><!--$NO-MVN-MAN-VER$ --> 
		</dependency> 
		<dependency> 
			<groupId>org.mybatis.spring.boot</groupId> 
			<artifactId>mybatis-spring-boot-starter</artifactId> 
			<version>1.1.1</version> 
		</dependency> 
		<!-- mysql 依赖 --> 
		<dependency> 
			<groupId>mysql</groupId> 
			<artifactId>mysql-connector-java</artifactId> 
		</dependency> 
		<!-- redis 依赖 --> 
		<dependency> 
            <groupId>org.springframework.boot</groupId> 
            <artifactId>spring-boot-starter-data-redis</artifactId> 
        </dependency> 
	</dependencies> 
   
</project> 

 service:

 

package com.toov5.service; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.cache.ehcache.EhCacheCacheManager; 
import org.springframework.stereotype.Component; 
 
import net.sf.ehcache.Cache; 
import net.sf.ehcache.Element; 
 
 
@Component 
public class EhCacheUtils { 
 
    // @Autowired 
    // private CacheManager cacheManager; 
    @Autowired 
    private EhCacheCacheManager ehCacheCacheManager; 
 
    // 添加本地缓存 (相同的key 会直接覆盖) 
    public void put(String cacheName, String key, Object value) { 
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); 
        Element element = new Element(key, value); 
        cache.put(element); 
    } 
 
    // 获取本地缓存 
    public Object get(String cacheName, String key) { 
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); 
        Element element = cache.get(key); 
        return element == null ? null : element.getObjectValue(); 
    } 
 
    public void remove(String cacheName, String key) { 
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); 
        cache.remove(key); 
    } 
 
}
package com.toov5.service; 
 
import java.util.Set; 
import java.util.concurrent.TimeUnit; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.data.redis.core.StringRedisTemplate; 
import org.springframework.stereotype.Component; 
 
@Component 
public class RedisService { 
 
    @Autowired 
    private StringRedisTemplate stringRedisTemplate; 
    //这样该方法支持多种数据类型  
    public void set(String key , Object object, Long time){ 
        //开启事务权限 
        stringRedisTemplate.setEnableTransactionSupport(true); 
        try { 
            //开启事务 
            stringRedisTemplate.multi(); 
             
            String argString =(String)object;  //强转下 
            stringRedisTemplate.opsForValue().set(key, argString); 
             
            //成功就提交 
            stringRedisTemplate.exec(); 
        } catch (Exception e) { 
            //失败了就回滚 
            stringRedisTemplate.discard(); 
             
        } 
        if (object instanceof String ) {  //判断下是String类型不 
            String argString =(String)object;  //强转下 
            //存放String类型的 
            stringRedisTemplate.opsForValue().set(key, argString); 
        } 
        //如果存放Set类型 
        if (object instanceof Set) { 
            Set<String> valueSet =(Set<String>)object; 
            for(String string:valueSet){ 
                stringRedisTemplate.opsForSet().add(key, string);  //此处点击下源码看下 第二个参数可以放好多 
            } 
        } 
        //设置有效期 
        if (time != null) { 
            stringRedisTemplate.expire(key, time, TimeUnit.SECONDS); 
        } 
         
    } 
    //做个封装 
    public void setString(String key, Object object){ 
        String argString =(String)object;  //强转下 
        //存放String类型的 
        stringRedisTemplate.opsForValue().set(key, argString); 
    } 
    public void setSet(String key, Object object){ 
        Set<String> valueSet =(Set<String>)object; 
        for(String string:valueSet){ 
            stringRedisTemplate.opsForSet().add(key, string);  //此处点击下源码看下 第二个参数可以放好多 
        } 
    } 
     
    public String getString(String key){ 
     return    stringRedisTemplate.opsForValue().get(key); 
    } 
     
     
}
package com.toov5.service; 
 
import java.util.concurrent.locks.Lock; 
import java.util.concurrent.locks.ReentrantLock; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.stereotype.Service; 
import org.springframework.web.bind.annotation.RequestMapping; 
 
import com.toov5.entity.Users; 
import com.toov5.mapper.UserMapper; 
 
import io.netty.util.internal.StringUtil; 
 
@Service 
public class SnowslideService { 
    @Autowired 
    private UserMapper userMapper;  
    @Autowired 
    private RedisService redisService; 
     
    private Lock lock = new ReentrantLock(); 
     
    public String getUser01(Long id){ 
    //定义key, key以当前的类名+方法名+id+参数值 
    String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() 
                        + "-id:" + id;     
          //1查询redis 
    String username = redisService.getString(key); 
    if (!StringUtil.isNullOrEmpty(username)) { 
        return username; 
    } 
    String resultUsaerName = null; 
    try { 
        //开启锁 
        lock.lock(); 
        Users user = userMapper.getUser(id); 
        if (username == null) { 
            return null; 
        } 
        resultUsaerName =user.getName(); 
        redisService.setString(key, resultUsaerName); 
    } catch (Exception e) { 
        // TODO: handle exception 
    }finally { 
        //释放锁 
        lock.unlock(); 
    } 
          //3直接返回 
    return resultUsaerName; 
    } 
     
     
     
//穿透解决方案     
    public String getUser02(Long id){ 
    //定义key, key以当前的类名+方法名+id+参数值 
    String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() 
                        + "-id:" + id;     
          //1查询redis 
    System.out.println("查询redis缓存"+"key"+key+".resultUserName"); 
    String username = redisService.getString(key); 
    if (!StringUtil.isNullOrEmpty(username)) { 
        return username; 
    } 
    String resultUsaerName = null; 
    //如果数据库中,没有对应的数据信息的时候 
       System.out.println("查询数据库:id"+id); 
        Users user = userMapper.getUser(id); 
        if (user == null) { 
            resultUsaerName="${null}";  //做个标记  客户端识别到后 提示下吧 
             
        }else { 
            resultUsaerName=user.getName(); 
        } 
        System.out.println("写入redis缓存"+"key"+key+".resultUserName"+resultUsaerName); 
        redisService.setString(key, resultUsaerName); 
 
          //3直接返回 
    return resultUsaerName; 
    } 
     
     
}
package com.toov5.service; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.stereotype.Component; 
 
import com.alibaba.fastjson.JSONObject; 
import com.toov5.entity.Users; 
import com.toov5.mapper.UserMapper; 
 
 
import io.netty.util.internal.StringUtil; 
 
@Component 
public class UserService { 
    @Autowired 
    private EhCacheUtils ehCacheUtils; 
    @Autowired 
    private RedisService redisService; 
    @Autowired 
    private UserMapper userMapper; 
    //定义个全局的cache名字 
    private String cachename ="userCache"; 
     
    public Users getUser(Long id){ 
        //先查询一级缓存  key以当前的类名+方法名+id+参数值 
        String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() 
                + "-id:" + id; 
        //查询一级缓存数据有对应值的存在 如果有 返回 
        Users user = (Users)ehCacheUtils.get(cachename, key); 
        if (user != null) { 
            System.out.println("key"+key+",直接从一级缓存获取数据"+user.toString()); 
            return user; 
        } 
        //一级缓存没有对应的值存在,接着查询二级缓存     
        // redis存对象的方式  json格式 然后反序列号 
        String userJson = redisService.getString(key); 
        //如果rdis缓存中有这个对应的值,修改一级缓存    最下面的会有的 相同会覆盖的     
        if (!StringUtil.isNullOrEmpty(userJson)) {  //有 转成json 
            JSONObject jsonObject = new JSONObject();//用的fastjson 
            Users resultUser = jsonObject.parseObject(userJson,Users.class); 
            ehCacheUtils.put(cachename, key, resultUser); 
            return resultUser; 
        } 
        //都没有 查询DB  
        Users user1 = userMapper.getUser(id); 
        if (user1 == null) { 
            return null; 
        } 
        //保证两级缓存有效期相同!?   一级缓存时间-二级缓存执行的时间 
        //一级缓存时间 等于 二级缓存剩下的时间    
        //存放到二级缓存 redis中 
        redisService.setString(key, new JSONObject().toJSONString(user1)); 
        //存放到一级缓存 Ehchache 
        ehCacheUtils.put(cachename, key, user1); 
        return user1; 
    } 
     
     
     
}

mapper

package com.toov5.mapper; 
 
import java.util.List; 
 
import org.apache.ibatis.annotations.Param; 
import org.apache.ibatis.annotations.Select; 
import org.springframework.cache.annotation.CacheConfig; 
import org.springframework.cache.annotation.Cacheable; 
 
import com.toov5.entity.Users; 
//引入的jar包后就有了这个注解了 非常好用 (配置缓存的基本信息) 
@CacheConfig(cacheNames={"userCache"})  //缓存的名字  整个类的 
public interface UserMapper { 
    @Select("SELECT ID ,NAME,AGE FROM users where id=#{id}") 
    @Cacheable //让这个方法实现缓存 查询完毕后 存入到缓存中  不是每个方法都需要缓存呀!save()就不用了吧 
    Users getUser(@Param("id") Long id); 
}

entity

package com.toov5.entity; 
 
import java.io.Serializable; 
 
import lombok.Data; 
 
@Data 
public class Users implements Serializable{ 
  private String name; 
  private Integer age; 
}

controller

package com.toov5.controller; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.web.bind.annotation.RequestMapping; 
import org.springframework.web.bind.annotation.RestController; 
 
import com.toov5.service.SnowslideService; 
 
@RestController 
public class UserRedisController { 
   @Autowired 
   private SnowslideService snowslideService; 
    
   @RequestMapping("/getUser02") 
   public String getUser02(Long id){ 
       return snowslideService.getUser02(id);  
   } 
    
}
package com.toov5.controller; 
 
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.web.bind.annotation.RequestMapping; 
import org.springframework.web.bind.annotation.RestController; 
 
import com.toov5.entity.Users; 
import com.toov5.service.UserService; 
 
@RestController 
public class IndexController { 
    @Autowired 
    private UserService userService; 
     
    @RequestMapping("/userId") 
    public Users getUserId(Long id){ 
        return userService.getUser(id);   
    } 
     
    
}

启动类

package com.toov5.app; 
 
import org.mybatis.spring.annotation.MapperScan; 
import org.springframework.boot.SpringApplication; 
import org.springframework.boot.autoconfigure.SpringBootApplication; 
import org.springframework.cache.annotation.EnableCaching; 
 
@EnableCaching //开启缓存 
@MapperScan(basePackages={"com.toov5.mapper"}) 
@SpringBootApplication(scanBasePackages={"com.toov5.*"}) 
public class app { 
   public static void main(String[] args) { 
    SpringApplication.run(app.class, args); 
} 
     
}

yml

###端口号配置 
server: 
  port: 8080 
###数据库配置  
spring: 
  datasource: 
    url: jdbc:mysql://localhost:3306/test 
    username: root 
    password: root 
    driver-class-name: com.mysql.jdbc.Driver 
    test-while-idle: true 
    test-on-borrow: true 
    validation-query: SELECT 1 FROM DUAL 
    time-between-eviction-runs-millis: 300000 
    min-evictable-idle-time-millis: 1800000 
# 缓存配置读取 
  cache: 
    type: ehcache 
    ehcache: 
      config: classpath:app1_ehcache.xml 
  redis: 
    database: 0  
    jedis: 
      pool: 
        max-active: 8 
        max-wait: -1 
        max-idle: 8 
        min-idle: 0 
    timeout: 10000 
    cluster: 
      nodes: 
        - 192.168.91.5:9001 
        - 192.168.91.5:9002 
        - 192.168.91.5:9003 
        - 192.168.91.5:9004 
        - 192.168.91.5:9005 
        - 192.168.91.5:9006 

 

<?xml version="1.0" encoding="UTF-8"?> 
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"> 
 
	<diskStore path="java.io.tmpdir/ehcache-rmi-4000" /> 
 
 
	<!-- 默认缓存 --> 
	<defaultCache maxElementsInMemory="1000" eternal="true" 
		timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true" 
		diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000" 
		diskPersistent="true" diskExpiryThreadIntervalSeconds="120" 
		memoryStoreEvictionPolicy="LRU"> 
	</defaultCache> 
   
	<!-- demo缓存 --><!-- name="userCache" 对应我们在 @CacheConfig(cacheNames={"userCache"}) !!!!! --> 
	<!--Ehcache底层也是用Map集合实现的 --> 
	<cache name="userCache" maxElementsInMemory="1000" eternal="false" 
		timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true" 
		diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000" 
		diskPersistent="false" diskExpiryThreadIntervalSeconds="120" 
		memoryStoreEvictionPolicy="LRU">  <!-- LRU缓存策略 --> 
		<cacheEventListenerFactory 
			class="net.sf.ehcache.distribution.RMICacheReplicatorFactory" /> 
		<!-- 用于在初始化缓存,以及自动设置 --> 
		<bootstrapCacheLoaderFactory 
			class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" /> 
	</cache> 
</ehcache> 

 再加一个拦截

Redis穿透问题解决方案详解编程语言

 

运行结果:

 Redis穿透问题解决方案详解编程语言

Redis穿透问题解决方案详解编程语言

 

把空结果,也给缓存起来,这样下次同样的请求就可以直接返回空了,即可以避免当查询的值为空时引起的缓存穿透。同时也可以单独设置个缓存区域存储空值,对要查询的key进行预先校验,然后再放行给后面的正常缓存处理逻辑。

 

注意:再给对应的ip存放真值的时候,需要先清除对应的之前的空缓存。

 

补充热点key

 

热点key:某个key访问非常频繁,当key失效的时候有打量线程来构建缓存,导致负载增加,系统崩溃。

 

解决办法:

①使用锁,单机用synchronized,lock等,分布式用分布式锁。

②缓存过期时间不设置,而是设置在key对应的value里。如果检测到存的时间超过过期时间则异步更新缓存。

③在value设置一个比过期时间t0小的过期时间值t1,当t1过期的时候,延长t1并做更新缓存操作。

 

 

 

 

 

 

 

原创文章,作者:Maggie-Hunter,如若转载,请注明出处:https://blog.ytso.com/16142.html

(0)
上一篇 2021年7月19日
下一篇 2021年7月19日

相关推荐

发表回复

登录后才能评论