Netty堆缓存问题

Alexandra ·

更新时间:2024-11-14

· 861 次阅读

Netty堆缓存问题1、问题描述2、问题分析3、问题解决4、总结 1、问题描述

今天学习Netty堆缓存和直接缓存遇到一个问题，明明使用的是堆缓存，这么读取不到数据呢？打印日志一看heapBuf.hasArray()直接返回false。来下面我们来看看源码，到底是怎么回事。

2、问题分析

首先写一个测试方法，直接向ByteBuf写入中国万岁!，然后如果是堆内存直接打印即可。源码如下：


    @Test
    public void testHeapBuffer2() {
        //取得堆内存 (但是默认是 directByDefault=true)
        ByteBuf heapBuf = ByteBufAllocator.DEFAULT.buffer();
        heapBuf.writeBytes("中国万岁！".getBytes(UTF_8));
        if (heapBuf.hasArray()) {
            //取得内部数组
            byte[] array = heapBuf.array();
            int offset = heapBuf.arrayOffset() + heapBuf.readerIndex();
            int length = heapBuf.readableBytes();
            Logger.info("---------chen------------>" + new String(array, offset, length, UTF_8));
        }
        System.out.println("heapBuf.hasArray(): " + heapBuf.hasArray());
        heapBuf.release();
    }

控制台输出如下：

09:55:28.154 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
09:55:28.213 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
09:55:28.216 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
09:55:28.217 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
09:55:28.217 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
09:55:28.218 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: Windows
09:55:28.218 [main] DEBUG io.netty.util.internal.PlatformDependent - Java version: 8
09:55:28.218 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
09:55:28.218 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
09:55:28.219 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
09:55:28.221 [main] DEBUG io.netty.util.internal.PlatformDependent - Javassist: unavailable
09:55:28.221 [main] DEBUG io.netty.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
09:55:28.222 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\ADMINI~1\AppData\Local\Temp (java.io.tmpdir)
09:55:28.222 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
09:55:28.222 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
09:55:28.225 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
09:55:28.226 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
09:55:28.228 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
09:55:28.241 [main] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.bytebuf.checkAccessible: true
09:55:28.245 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
09:55:28.245 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.maxRecords: 4
heapBuf.hasArray(): false
09:55:28.253 [main] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available

是的，我们看到了heapBuf.hasArray(): false。ByteBufAllocator.DEFAULT.buffer();不是使用的是堆缓存吗？怎么会是false呢？纸上得来终觉浅，绝知此事要躬行。直接看源码。他的实现方法如下：

    @Override
    public ByteBuf buffer() {
        if (directByDefault) {
            return directBuffer();
        }
        return heapBuffer();
    }

嗯，到底是堆内存还是直接内存还和directByDefault变量有关。我们在查找这个变量是在哪里赋值的。没错就是他的构造函数。

    /**
     * Create new instance
     *
     * @param preferDirect {@code true} if {@link #buffer(int)} should try to allocate a direct buffer rather than
     *                     a heap buffer
     */
    protected AbstractByteBufAllocator(boolean preferDirect) {
        directByDefault = preferDirect && PlatformDependent.hasUnsafe();
        emptyBuf = new EmptyByteBuf(this);
    }

是的，我们还没有找到答案，因为他还是和preferDirect变量相关。继续往下找。我们发现UnpooledByteBufAllocator的构造函数设置了这个值。

    /**
     * Create a new instance
     *
     * @param preferDirect {@code true} if {@link #buffer(int)} should try to allocate a direct buffer rather than
     *                     a heap buffer
     */
    public UnpooledByteBufAllocator(boolean preferDirect) {
        super(preferDirect);
    }

继续寻找preferDirect的值。

  /**
     * Default instance
     */
    public static final UnpooledByteBufAllocator DEFAULT =
            new UnpooledByteBufAllocator(PlatformDependent.directBufferPreferred());

真实值是PlatformDependent.directBufferPreferred()，再次追踪进去。

    /**
     * Returns {@code true} if the platform has reliable low-level direct buffer access API and a user has not specified
     * {@code -Dio.netty.noPreferDirect} option.
     */
    public static boolean directBufferPreferred() {
        return DIRECT_BUFFER_PREFERRED;
    }

这个DIRECT_BUFFER_PREFERRED是何方神圣呢？继续追踪。

    private static final boolean DIRECT_BUFFER_PREFERRED =
            HAS_UNSAFE && !SystemPropertyUtil.getBoolean("io.netty.noPreferDirect", false);

终于找到了作妖的源头了。是的就是他。根据定义我们知道DIRECT_BUFFER_PREFERRED默认是True，也就是默认是直接内存。OMG

3、问题解决

既然知道你是直接内存了，那我们直接改属性不就完了。通过System.setProperty("io.netty.noPreferDirect", "true");直接设置使用堆缓存。修改代码如下：

    //堆缓冲区
    @Test
    public void testHeapBuffer() {
        System.setProperty("io.netty.noPreferDirect", "true");
        //取得堆内存 (但是默认是 directByDefault=true)
        ByteBuf heapBuf = ByteBufAllocator.DEFAULT.buffer();
        heapBuf.writeBytes("中国万岁！".getBytes(UTF_8));
        if (heapBuf.hasArray()) {
            //取得内部数组
            byte[] array = heapBuf.array();
            int offset = heapBuf.arrayOffset() + heapBuf.readerIndex();
            int length = heapBuf.readableBytes();
            Logger.info("---------chen------------>" + new String(array, offset, length, UTF_8));
        }
        heapBuf.release();
    }

直接运行，查看控制台

10:11:36.587 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
10:11:36.636 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
10:11:36.638 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
10:11:36.639 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
10:11:36.639 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
10:11:36.640 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: Windows
10:11:36.640 [main] DEBUG io.netty.util.internal.PlatformDependent - Java version: 8
10:11:36.640 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
10:11:36.640 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
10:11:36.641 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
10:11:36.642 [main] DEBUG io.netty.util.internal.PlatformDependent - Javassist: unavailable
10:11:36.642 [main] DEBUG io.netty.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
10:11:36.643 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\ADMINI~1\AppData\Local\Temp (java.io.tmpdir)
10:11:36.643 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
10:11:36.643 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: true
10:11:36.646 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
10:11:36.647 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
10:11:36.649 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
10:11:36.659 [main] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.bytebuf.checkAccessible: true
10:11:36.664 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
10:11:36.664 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.maxRecords: 4
[main|BufferTypeTest.testHeapBuffer] |>  ---------chen------------>中国万岁！

[main|BufferTypeTest.testHeapBuffer] |> ---------chenwei------------>中国万岁！完美输出结果。问题解决。

4、总结

书上的代码直接运行绝大部分是对的，但是总有一些软件的更新使得作者无能为力。之前的API是对的，但是之后就废弃了或修改了是常有的事。所以我们需要跟踪源代码。这只是一个小小的问题，如果没有前辈的无私奉献，很难想象我们自己一天能学到多少内容。感谢各位前辈的辛勤付出，让我们少走了很多的弯路！

点个赞再走呗！欢迎留言哦！

作者：武汉加油、中国加油

netty 缓存

1024 个赞