作者：微信小助手
<p data-mpa-powered-by="yiban.io"><em style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-style: italic;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-align: justify;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;text-decoration-thickness: initial;text-decoration-style: initial;text-decoration-color: initial;background-color: rgb(255, 255, 255);visibility: visible;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;letter-spacing: 0.544px;font-size: 16px;text-align: left;color: rgb(0, 0, 0);font-family: 微软雅黑;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 15px;visibility: visible;">点击关注公众号，实用技术文章</span></strong><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;letter-spacing: 0.544px;font-size: 16px;text-align: left;color: rgb(123, 12, 0);visibility: visible;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: 微软雅黑;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 15px;visibility: visible;">及时了解</span></strong></span><img class="rich_pages wxw-img" data-fileid="100031040" data-ratio="1" data-type="png" data-w="64" src="/upload/29316807de8eec165a101cfe6173a39c.png" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;vertical-align: bottom;height: auto !important;letter-spacing: 0.544px;font-size: 16px;text-align: left;color: rgb(0, 0, 0);font-family: 微软雅黑;visibility: visible !important;max-height: 20px !important;width: 20px !important;"></em></p> <section class="mp_profile_iframe_wrp"> <mpprofile class="js_uneditable custom_select_card mp_profile_iframe" data-pluginname="mpprofile" data-id="MzI4Njc5NjM1NQ==" data-headimg="http://mmbiz.qpic.cn/mmbiz_png/eQPyBffYbueAgIuCqZdZnW3NW44AOD32W2BOe28vCWLC2XdcNqJufjmlCCI2YVbFh0fjL6qCxEoNjHN9jTBItQ/0?wx_fmt=png" data-nickname="Java知音" data-alias="Java_friends" data-signature="专注于java。分享java基础、原理性知识、JavaWeb实战、spring全家桶、设计模式及面试资料、开源项目，助力开发者成长！" data-from="0"></mpprofile> </section> <section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style=""> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;"><span style="display: none;"></span>前言<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">组内许多服务既有同步接口也有异步脚本，接口和脚本的日志都打印在同一个日志文件中，日志繁杂给排查问题带来不少的阻碍。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">为了解决这个问题，同事提了个按照接口分类日志文件的技术需求，也就是一个同步接口对应一个日志文件，从而将日志区分开。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">目前组内所有服务都是使用 logback 日志框架，笔者对这个需求产生了一定的兴趣，查找资料了解到了 logback 日志过滤器，因此有了本文，读者有兴趣的话也可以去官方文档：</p> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;display: block;font-size: 0.9em;overflow: auto;border-left: 3px solid rgba(0, 0, 0, 0.4);background: rgba(0, 0, 0, 0.05);color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0px;color: black;line-height: 26px;">https://logback.qos.ch/manual/filters.html</p> </blockquote> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;"><span style="display: none;"></span>1. Logback 过滤器的分类<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">logback 提供两种类型的过滤器, 一种是常规过滤器(regular filters) ，另一种是全局过滤器(turbo filter)。常规过滤器与 appender 绑定， 全局过滤器与 logger context 绑定，二者的区别就是全局过滤器过滤所有 logging request ，而常规过滤器只过滤某个 appender 的 logging request</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;"><span style="display: none;"></span>2. 常规过滤器<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">在 logback-classic 中常规过滤器继承 ch.qos.logback.core.filter.Filter 抽象类，该抽象类的 decide()抽象方法接收一个 ILoggingEvent 参数，返回一个FilterReply枚举值。枚举值标明了过滤器对当前日志事件的过滤情况，具体处理如下表</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.20150187734668334" src="/upload/baa1349d383d169343b285f7af5cf650.png" data-type="png" data-w="799" style="display: block;margin: 0 auto;max-width: 100%;"> </figure> <h4 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 18px;"><span style="display: none;"></span>2.1 日志级别过滤器 LevelFilter<span style="display: none;"></span></h4> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">LevelFilter 的过滤是基于日志事件的级别，如果日志级别等于配置的 level，则过滤器通过，否则拒绝，其代码实现如下。另外ThresholdFilter也是基于日志等级门槛过滤的，只不过其逻辑是当日志级别大于等于配置等级才能通过过滤器，此处不再赘述</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">需注意源码中涉及的成员变量 this.level、this.onMatch以及this.onMismatch都是在过滤器初始化时根据配置的值自动注入的，代码中只要提供其 set 方法即可</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;"><code style="overflow-x: auto;padding: 16px;background: #272822;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;-webkit-overflow-scrolling: touch;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">public</span> <span style="line-height: 26px;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">class</span> <span style="font-weight: bold;color: white;line-height: 26px;">LevelFilter</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">extends</span> <span style="font-weight: bold;color: white;line-height: 26px;">AbstractMatcherFilter</span><<span style="font-weight: bold;color: white;line-height: 26px;">ILoggingEvent</span>> </span>{<br> Level level;<br><br> <span style="line-height: 26px;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">public</span> <span style="color: #a6e22e;font-weight: bold;line-height: 26px;">LevelFilter</span><span style="line-height: 26px;">()</span> </span>{<br> }<br><br> <span style="line-height: 26px;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">public</span> FilterReply <span style="color: #a6e22e;font-weight: bold;line-height: 26px;">decide</span><span style="line-height: 26px;">(ILoggingEvent event)</span> </span>{<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span> (!<span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.isStarted()) {<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">return</span> FilterReply.NEUTRAL;<br> } <span style="color: #f92672;font-weight: bold;line-height: 26px;">else</span> {<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">return</span> event.getLevel().equals(<span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.level) ? <span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.onMatch : <span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.onMismatch;<br> }<br> }<br><br> <span style="line-height: 26px;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">public</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">void</span> <span style="color: #a6e22e;font-weight: bold;line-height: 26px;">setLevel</span><span style="line-height: 26px;">(Level level)</span> </span>{<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.level = level;<br> }<br><br> <span style="line-height: 26px;"><span style="color: #f92672;font-weight: bold;line-height: 26px;">public</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">void</span> <span style="color: #a6e22e;font-weight: bold;line-height: 26px;">start</span><span style="line-height: 26px;">()</span> </span>{<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span> (<span style="color: #f92672;font-weight: bold;line-height: 26px;">this</span>.level != <span style="color: #f92672;font-weight: bold;line-height: 26px;">null</span>) {<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">super</span>.start();<br> }<br><br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">以下为LevelFilter 配置示例，该配置需要关注的点如下：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;"><code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< appender ></code>标签配置指定的 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">RollingFileAppender</code> 为滚动文件追加器，其滚动策略由 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< rollingPolicy ></code>标签配置，指定为 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">TimeBasedRollingPolicy</code></p> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">常规过滤器需与 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">appender</code> 绑定，过滤器元素<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< filter ></code>需要在 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< appender ></code>标签之间。配置<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">LevelFilter</code> 的<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< filter ></code>标签中的 3 个子标签配置了需要注入 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">this.level</code>、<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">this.onMatch</code>以及<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">this.onMismatch</code> 属性的值，实现自定义过滤器时标签名可以自定义，只要与属性名相对应即可</p> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;"><code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< appender-ref ></code> 标签启用指定的 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">appender</code></p> </section></li> </ul> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;"><code style="overflow-x: auto;padding: 16px;background: #272822;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;-webkit-overflow-scrolling: touch;"><span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender</span> <span style="line-height: 26px;">name</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_ALL"</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.RollingFileAppender"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">filter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.classic.filter.LevelFilter"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">level</span>></span>INFO<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">level</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">onMatch</span>></span>ACCEPT<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">onMatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">onMismatch</span>></span>DENY<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">onMismatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">filter</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">rollingPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.TimeBasedRollingPolicy"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span>90<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span>${BASE_PATH_ERROR}/%d{yyyy-MM-dd}.%i.log.gz<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span>200MB<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">rollingPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">pattern</span>></span> %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{30} - %msg%n<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">pattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">appender</span>></span><br> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">root</span> <span style="line-height: 26px;">level</span>=<span style="color: #a6e22e;line-height: 26px;">"INFO"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender-ref</span> <span style="line-height: 26px;">ref</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_ALL"</span> /></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">root</span>></span><br> <br><span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br></code></pre> <h4 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 18px;"><span style="display: none;"></span>2.2 日志评估过滤器 EvaluatorFilter<span style="display: none;"></span></h4> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;"><code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">EvaluatorFilter</code> 是一个抽象类，它包含了一个 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">EventEvaluator</code> 对象，这个对象是日志评估过滤器实现的关键。它会封装配置在<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< expression ></code>标签中的过滤逻辑，在处理日志事件时负责过滤条件的判断</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;"><code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">EventEvaluator</code>主要有两个实现类，分别是 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">GEventEvaluator</code> 和 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">JaninoEventEvaluator</code>。<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">GEventEvaluator</code> 接收 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">Groovy</code> 语言的条件表达式作为判断条件，<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">JaninoEventEvaluator</code> 接收一个 java 的判断表达式作为判断条件，logback 默认使用的是 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">JaninoEventEvaluator</code>，其依赖于 Janino 相关库</p> <h4 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 18px;"><span style="display: none;"></span>JaninoEventEvaluator 的使用<span style="display: none;"></span></h4> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">使用 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">JaninoEventEvaluator</code>评估器时 logback 会自动导出日志事件对象的属性到 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">evaluation</code> 表达式中，所以可以直接使用以下属性</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.6105919003115264" src="/upload/6e62e48b702de5772902a67b0ce7efd8.png" data-type="png" data-w="963" style="display: block;margin: 0 auto;max-width: 100%;"> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;"><code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">JaninoEventEvaluator</code> 的使用配置如下所示，<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< expression ></code>标签包裹的表达式即为日志过滤的判断逻辑。根据其源代码实现来看，如果未检测到表达式中的 return 字符串将直接在表达式首尾添上 return 和 分号</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;"><code style="overflow-x: auto;padding: 16px;background: #272822;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;-webkit-overflow-scrolling: touch;"><span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender</span> <span style="line-height: 26px;">name</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_ALL"</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.RollingFileAppender"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">filter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.filter.EvaluatorFilter"</span>></span> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">evaluator</span>></span> <span style="color: #75715e;line-height: 26px;"><!-- defaults to type ch.qos.logback.classic.boolex.JaninoEventEvaluator --></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">expression</span>></span>return message.contains("nathan");<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">expression</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">evaluator</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMismatch</span>></span>NEUTRAL<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMismatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span>ACCEPT<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">filter</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">rollingPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.TimeBasedRollingPolicy"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span>90<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span>${BASE_PATH_ERROR}/%d{yyyy-MM-dd}.%i.log.gz<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span>200MB<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">rollingPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">pattern</span>></span> %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{30} - %msg%n<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">pattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">appender</span>></span><br><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">root</span> <span style="line-height: 26px;">level</span>=<span style="color: #a6e22e;line-height: 26px;">"INFO"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender-ref</span> <span style="line-height: 26px;">ref</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_ALL"</span> /></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">root</span>></span><br><br><span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br></code></pre> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;"><span style="display: none;"></span>3. 全局过滤器<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">在 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">logback-classic</code> 中全局过滤器都继承自抽象类 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">ch.qos.logback.classic.turbo.TurboFilter</code> ，其实 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">Trubo Filter</code> 与 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">Regular Filter</code> 只有两点主要的不同：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">TurboFilter 对象与 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">logging context</code> 绑定，因此它会处理所有的 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">logging request</code>，而不是单独的appender，过滤范围更广</p> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">TurboFilter 过滤器在 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">LoggingEvent</code> 对象创建之前就已经被调用，所以并不需要日志事件来过滤<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">logging request</code>，因此会有更好的性能表现</p> </section></li> </ul> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">logback 提供了一些常用的 TurboFilter:</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <strong style="font-weight: bold;color: black;">MDCFilter</strong> : 通过 MDC 过滤，主要校验 MDC 中指定 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">key-value</code> 是否存在 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <strong style="font-weight: bold;color: black;">DynamicThresholdFilter</strong> ：通过 MDC 或 level 过滤 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <strong style="font-weight: bold;color: black;">MarkerFilter</strong> ：通过 marker 标签过滤 </section></li> </ul> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">以下为全局过滤器使用的配置示例</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;"><code style="overflow-x: auto;padding: 16px;background: #272822;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;-webkit-overflow-scrolling: touch;"><span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">turboFilter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.classic.turbo.DynamicThresholdFilter"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">Key</span>></span>username<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">Key</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">DefaultThreshold</span>></span>DEBUG<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">DefaultThreshold</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">MDCValueLevelPair</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">value</span>></span>nathan<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">value</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">level</span>></span>DEBUG<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">level</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">MDCValueLevelPair</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">turboFilter</span>></span><br> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">turboFilter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.classic.turbo.MDCFilter"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">MDCKey</span>></span>username<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">MDCKey</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">Value</span>></span>nathan<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">Value</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span>ACCEPT<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">turboFilter</span>></span><br> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">turboFilter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.classic.turbo.MarkerFilter"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">Marker</span>></span>nathan<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">Marker</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span>DENY<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">turboFilter</span>></span><br><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender</span> <span style="line-height: 26px;">name</span>=<span style="color: #a6e22e;line-height: 26px;">"STDOUT"</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.ConsoleAppender"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">pattern</span>></span>%date [%thread] %-5level %logger - %msg%n<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">pattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">appender</span>></span><br><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">root</span> <span style="line-height: 26px;">level</span>=<span style="color: #a6e22e;line-height: 26px;">"INFO"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender-ref</span> <span style="line-height: 26px;">ref</span>=<span style="color: #a6e22e;line-height: 26px;">"STDOUT"</span>/></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">root</span>></span> <br> <br><span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br></code></pre> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;"><span style="display: none;"></span>4. 日志分类归档方案<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">按照接口的名称分类日志，将接口方法执行期间的所有日志输出到指定到接口文件中，其需要解决的主要问题有两个：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 怎么分辨哪些日志是哪个接口被调用过程中打印的? </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 接口的日志怎么输出到指定到文件？ </section></li> </ul> <h4 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 18px;"><span style="display: none;"></span>问题1方案<span style="display: none;"></span></h4> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">对于问题1，首先想到的就是对特定的日志打上特定的标记，可以参考文章 Slf4j 中的 MDC ，借助 MDC 把接口名称以一个 key 存储下来，则可据此区分不同的接口的日志</p> <h4 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 18px;"><span style="display: none;"></span>问题2方案<span style="display: none;"></span></h4> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">对于问题2，本文介绍的 logback 日志过滤器便起到了作用。我们可以为每个接口重新配置一个 appender，其关键点如下：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">使用 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">EvaluatorFilter</code> 过滤器，配置<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< expression ></code>标签表达式通过 MDC 获取指定 key 的 value 是否等于目标值</p> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">配置日志的滚动策略中 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">< fileNamePattern > </code>日志文件名标签为指定的格式</p> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: black;">配置<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;"> < appender-ref ></code> 标签启用指定的 appender</p> </section></li> </ul> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;"><code style="overflow-x: auto;padding: 16px;background: #272822;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;-webkit-overflow-scrolling: touch;"> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">configuration</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender</span> <span style="line-height: 26px;">name</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_METHOD"</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.RollingFileAppender"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">filter</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.filter.EvaluatorFilter"</span>></span> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">evaluator</span>></span> <span style="color: #75715e;line-height: 26px;"><!-- defaults to type ch.qos.logback.classic.boolex.JaninoEventEvaluator --></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">expression</span>></span>return ((String)mdc.get("key")).equals("method");<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">expression</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">evaluator</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMismatch</span>></span>NEUTRAL<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMismatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span>ACCEPT<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">OnMatch</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">filter</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">rollingPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.TimeBasedRollingPolicy"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span>90<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxHistory</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span>${BASE_PATH_ERROR}/%d{yyyy-MM-dd}.%i.log.method.gz<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">fileNamePattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span> <span style="line-height: 26px;">class</span>=<span style="color: #a6e22e;line-height: 26px;">"ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span>200MB<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">maxFileSize</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">timeBasedFileNamingAndTriggeringPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">rollingPolicy</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">pattern</span>></span> %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{30} - %msg%n<span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">pattern</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">encoder</span>></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">appender</span>></span><br> <br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">root</span> <span style="line-height: 26px;">level</span>=<span style="color: #a6e22e;line-height: 26px;">"INFO"</span>></span><br> <span style="color: #f92672;line-height: 26px;"><<span style="color: #f92672;line-height: 26px;">appender-ref</span> <span style="line-height: 26px;">ref</span>=<span style="color: #a6e22e;line-height: 26px;">"FILE_METHOD"</span> /></span><br> <span style="color: #f92672;line-height: 26px;"></<span style="color: #f92672;line-height: 26px;">root</span>></span><br></code></pre> </section> <section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="margin: 0px;padding: 0px 10px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;text-decoration-thickness: initial;text-decoration-style: initial;text-decoration-color: initial;background-color: rgb(255, 255, 255);font-size: 16px;color: black;line-height: 1.6;letter-spacing: 0px;word-break: break-word;text-align: left;font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;visibility: visible;"> <h3 data-tool="mdnice编辑器" style="margin: 0px;padding: 0px;outline: 0px;font-weight: 400;font-size: 16px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-align: right;visibility: visible;"><em style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-style: italic;color: rgb(136, 136, 136);font-size: 12px;letter-spacing: 0.5px;visibility: visible;">来源：https://blog.csdn.net/weixin_45505313</em></h3> </section>
作者：微信小助手
<h3 data-tool="mdnice编辑器" style="margin: 30px 0px 15px;padding: 0px;outline: 0px;font-weight: bold;font-size: 20px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;visibility: visible;">文章目录</h3> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;visibility: visible;">一、SQL语句及索引的优化</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;visibility: visible;">SQL语句的优化</p> <ol data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin: 8px 0px;padding: 0px 0px 0px 25px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;visibility: visible;"> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 尽量避免使用子查询 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 用IN来替换OR </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 读取适当的记录LIMIT M,N，而不要读多余的记录 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 禁止不必要的Order By排序 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 总和查询可以禁止排重用union all </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 避免随机取记录 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 将多次插入换成批量Insert插入 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 只返回必要的列，用具体的字段列表代替 select * 语句 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 区分in和exists </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);visibility: visible;"> 优化Group By语句 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 尽量使用数字型字段 </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 优化Join语句 </section></li> </ol> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">索引的优化/如何避免索引失效</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">二、数据库表结构的优化：使得数据库结构符合三大范式与BCNF</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">三、系统配置的优化</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">四、硬件的优化</p> <hr data-tool="mdnice编辑器" style="margin: 10px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;height: 1px;border-right: none;border-bottom: none;border-left: none;border-top-style: solid;border-top-color: black;"> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">在开始介绍如何优化sql前，先附上mysql内部逻辑图让大家有所了解</p> <figure data-tool="mdnice编辑器" style="margin: 10px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.6350710900473934" src="/upload/3ebcc1e13cca0e8c0352c4dab75113ad.png" data-type="png" data-w="1055" style="margin: 0px auto;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;vertical-align: bottom;height: auto !important;display: block;visibility: visible !important;width: 556px !important;"> </figure> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;">（1）连接器：</strong> 主要负责跟客户端建立连接、获取权限、维持和管理连接</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;">（2）查询缓存：</strong> 优先在缓存中进行查询，如果查到了则直接返回，如果缓存中查询不到，在去数据库中查询。</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">MySQL缓存是默认关闭的，也就是说不推荐使用缓存，并且在MySQL8.0 版本已经将查询缓存的整块功能删掉了。这主要是它的使用场景限制造成的：</p> <ul data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin: 8px 0px;padding: 0px 0px 0px 25px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 先说下缓存中数据存储格式：key（sql语句）- value（数据值），所以如果SQL语句（key）只要存在一点不同之处就会直接进行数据库查询了； </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 由于表中的数据不是一成不变的，大多数是经常变化的，而当数据库中的数据变化了，那么相应的与此表相关的缓存数据就需要移除掉； </section></li> </ul> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;">（3）解析器/分析器：</strong> 分析器的工作主要是对要执行的SQL语句进行词法解析、语法解析，最终得到抽象语法树，然后再使用预处理器对抽象语法树进行语义校验，判断抽象语法树中的表是否存在，如果存在的话，在接着判断select投影列字段是否在表中存在等。</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;">（4）优化器：</strong> 主要将SQL经过词法解析、语法解析后得到的语法树，通过数据字典和统计信息的内容，再经过一系列运算 ，最终得出一个执行计划，包括选择使用哪个索引</p> <ul data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin: 8px 0px;padding: 0px 0px 0px 25px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 在分析是否走索引查询时，是通过进行动态数据采样统计分析出来；只要是统计分析出来的，那就可能会存在分析错误的情况，所以在SQL执行不走索引时，也要考虑到这方面的因素 </section></li> </ul> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;">（5）执行器：</strong> 根据一系列的执行计划去调用存储引擎提供的API接口去调用操作数据，完成SQL的执行。</p> <h3 data-tool="mdnice编辑器" style="margin: 30px 0px 15px;padding: 0px;outline: 0px;font-weight: bold;font-size: 20px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;">一、SQL语句及索引的优化</h3> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">SQL语句的优化</p> <h5 data-tool="mdnice编辑器" style="margin: 30px 0px 15px;padding: 0px;outline: 0px;font-weight: bold;font-size: 16px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;">1. 尽量避免使用子查询</h5> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">例：</p> <pre data-tool="mdnice编辑器" style="margin: 10px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"><code style="margin: 0px;padding: 16px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;">SELECT * FROM t1 WHERE id (SELECT id FROM t2 WHERE name = <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(166, 226, 46);line-height: 26px;">'chackca'</span>);<br style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"></code></pre> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">其子查询在Mysql5.5版本里，内部执行计划是这样：先查外表再匹配内表，而不是先查内表t2，当外表的数据很大时，查询速度会非常慢。</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">在MariaDB10/Mysql5.6版本里，采用join关联方式对其进行了优化，这条SQL语句会自动转换为：<code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">SELECT t1.* FROM t1 JOIN t2 on t1.id = t2.id</code></p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">但请注意的是：优化只针对SELECT有效，对<code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">UPDATE/DELETE</code>子查询无效，固生产环境应避免使用子查询</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">由于MySQL的优化器对于子查询的处理能力比较弱，所以不建议使用子查询，可以改写成<code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Inner Join</code>，之所以 join 连接效率更高，是因为 MySQL不需要在内存中创建临时表</p> <h5 data-tool="mdnice编辑器" style="margin: 30px 0px 15px;padding: 0px;outline: 0px;font-weight: bold;font-size: 16px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;">2. 用IN来替换OR</h5> <ul data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin: 8px 0px;padding: 0px 0px 0px 25px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 低效查询： <code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">SELECT * FROM t WHERE id = 10 OR id = 20 OR id = 30;</code> </section></li> <li style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin: 5px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;line-height: 26px;color: rgb(1, 1, 1);"> 高效查询：S <code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">ELECT * FROM t WHERE id IN (10,20,30);</code> </section></li> </ul> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">另外，MySQL对于IN做了相应的优化，即将IN中的常量全部存储在一个数组里面，而且这个数组是排好序的。但是如果数值较多，产生的消耗也是比较大的。再例如：<code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">select id from table_name where num in(1,2,3) </code>对于连续的数值，能用 between 就不要用 in 了；再或者使用连接来替换。</p> <h5 data-tool="mdnice编辑器" style="margin: 30px 0px 15px;padding: 0px;outline: 0px;font-weight: bold;font-size: 16px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;">3. 读取适当的记录LIMIT M,N，而不要读多余的记录</h5> <pre data-tool="mdnice编辑器" style="margin: 10px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"><code style="margin: 0px;padding: 16px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;">select id,name from t <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(166, 226, 46);line-height: 26px;">limit</span> 866613, 20<br style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"></code></pre> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">使用上述sql语句做分页的时候，可能有人会发现，随着表数据量的增加，直接使用limit分页查询会越来越慢。</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">对于 <code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">limit m, n</code> 的分页查询，越往后面翻页（即m越大的情况下）SQL的耗时会越来越长，对于这种应该先取出主键id，然后通过主键id跟原表进行Join关联查询。因为MySQL 并不是跳过 offset 行，而是取 <code style="margin: 0px 2px;padding: 2px 4px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">offset+N</code> 行，然后放弃前 offset 行，返回 N 行，那当 offset 特别大的时候，效率就非常的低下，要么控制返回的总页数，要么对超过特定阈值的页数进行 SQL 改写。</p> <p data-tool="mdnice编辑器" style="margin: 0px;padding: 8px 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;line-height: 26px;">优化的方法如下：可以取前一页的最大行数的id（将上次遍历到的最末尾的数据ID传给数据库，然后直接定位到该ID处，再往后面遍历数据），然后根据这个最大的id来限制下一页的起点。比如此列中，上一页最大的id是866612。sql可以采用如下的写法：</p> <pre data-tool="mdnice编辑器" style="margin: 10px 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(0, 0, 0);font-size: 16px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: normal;orphans: 2;text-align: left;text-indent: 0px;text-transform: none;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;"><code style="margin: 0px;padding: 16px;outline: 0px
作者：微信小助手
<section data-class="_mbEditor" data-id="132251" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"> <section style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: Arial;border-width: 0px;border-style: none;border-color: initial;color: rgb(49, 147, 105);visibility: visible;"> <section class="mp_profile_iframe_wrp"> <mpprofile class="js_uneditable custom_select_card mp_profile_iframe" data-pluginname="mpprofile" data-id="MzkyNzExODM3OA==" data-headimg="http://mmbiz.qpic.cn/mmbiz_png/FibLXRKztelWZcHdUd1gtkSCYxXrd3Hj45GpTLdljGiaykvo8dyHibqO0hIfiad3lqFo7wmoamWfRCPat8HLicm87kw/0?wx_fmt=png" data-nickname="Java仓库" data-alias="" data-signature="专注Java全栈开发，分享实用技术干货。" data-from="0"></mpprofile> </section> <section data-class="_mbEditor" data-id="132251" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"> <section style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: Arial;border-width: 0px;border-style: none;border-color: initial;color: rgb(49, 147, 105);visibility: visible;"> <section style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;display: inline-block;float: left;color: rgb(255, 255, 255);border-color: rgb(226, 121, 97);background-color: rgb(226, 121, 97);visibility: visible;"> <section style="margin: 0px;padding: 5px 8px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;display: inline-block;vertical-align: middle;color: inherit;visibility: visible;"> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: inherit;visibility: visible;">1<span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;visibility: visible;"></span></p> </section> </section> <section style="margin: 0px;padding: 2px 0px 0px 40px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 18px;visibility: visible;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;font-family: 微软雅黑;visibility: visible;">前言</span> </section> <section style="margin: 0px;padding: 2px 0px 0px 40px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 18px;visibility: visible;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;font-family: 微软雅黑;visibility: visible;"><br></span> </section> </section> </section> <section style="margin: 0px;padding: 2px 0px 0px 40px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 18px;visibility: visible;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;font-family: 微软雅黑;visibility: visible;"></span> </section> </section> </section> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 15px;visibility: visible;">我们开发中经常用到Redis作为缓存，将高频数据放在Redis中能够提高业务性能，降低MySQL等关系型数据库压力，甚至一些系统使用Redis进行数据持久化，Redis松散的文档结构非常适合业务系统开发，在精确查询，数据统计业务有着很大的优势。</span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 15px;visibility: visible;">但是高频数据流处理系统中，Redis的压力也会很大，同时I/0开销才是耗时的主要原因，这时候为了降低Redis读写压力我们可以用到本地缓存，Guava为我们提供了优秀的本地缓存API，包含了过期策略等等，编码难度低，个人非常推荐。</span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"></p> <section data-class="_mbEditor" data-id="132251" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"> <section style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: Arial;border-width: 0px;border-style: none;border-color: initial;color: rgb(49, 147, 105);visibility: visible;"> <section style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;display: inline-block;float: left;color: rgb(255, 255, 255);border-color: rgb(226, 121, 97);background-color: rgb(226, 121, 97);visibility: visible;"> <section style="margin: 0px;padding: 5px 8px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;display: inline-block;vertical-align: middle;color: inherit;visibility: visible;"> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: inherit;visibility: visible;">2<span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;visibility: visible;"></span></p> </section> </section> <section style="margin: 0px;padding: 2px 0px 0px 40px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 18px;visibility: visible;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: inherit;font-family: 微软雅黑;visibility: visible;">设计示例</span> </section> </section> </section> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);visibility: visible;"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);font-size: 15px;visibility: visible;">Redis懒加载缓存</span></strong></span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 15px;visibility: visible;">数据在新增到MySQL不进行缓存，在精确查找进行缓存，做到查询即缓存，不查询不缓存</span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;visibility: visible;"><br></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);font-size: 15px;">流程图</span></strong></span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"><br></p> <p style="margin: 0px 0px 0em;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: center;"><img class="rich_pages wxw-img" data-ratio="0.534375" src="/upload/f96de4b266f23e8c81a4b3bf66945efa.jpg" data-type="jpeg" data-w="640" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;vertical-align: bottom;height: auto !important;float: none;width: 640px !important;visibility: visible !important;"></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"><br style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);"><strong style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(255, 76, 0);font-size: 15px;">代码示例</span></strong></span></p> <p style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;clear: both;min-height: 1em;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"><br></p> <section data-mpa-preserve-tpl-color="t" data-mpa-template="t" mpa-preserve="t" mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;color: rgb(34, 34, 34);font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 17px;font-style: normal;font-variant-ligatures: normal;font-variant-caps: normal;font-weight: 400;letter-spacing: 0.544px;orphans: 2;text-indent: 0px;text-transform: none;white-space: normal;widows: 2;word-spacing: 0px;-webkit-text-stroke-width: 0px;background-color: rgb(255, 255, 255);text-decoration-style: initial;text-decoration-color: initial;text-align: left;"> <pre style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;background: none;"><code style="margin: 0px 0.15em;padding: 5.95px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;border-radius: 4px;font-size: 0.85em;background: rgb(35, 36, 31);color: rgb(248, 248, 242);display: block;overflow-x: auto;white-space: nowrap;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 12px;font-family: CourierNewPS-ItalicMT;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(117, 113, 94);background: rgba(0, 0, 0, 0);display: inline;width: 316px;text-decoration: none solid rgb(117, 113, 94);font-weight: 400;font-style: normal;">// 伪代码示例 Xx代表你的的业务对象 如User Goods等等</span><br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 39px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">public</span> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 33px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">class</span> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(166, 226, 46);background: rgba(0, 0, 0, 0);display: inline;width: 72px;text-decoration: none solid rgb(166, 226, 46);font-weight: 400;font-style: normal;">XxLazyCache</span> {<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> @Autowired<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 46px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">private</span> RedisTemplate<String, Xx> redisTemplate;<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> @Autowired<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 46px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">private</span> XxService xxService;<span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(117, 113, 94);background: rgba(0, 0, 0, 0);display: inline;width: 114px;text-decoration: none solid rgb(117, 113, 94);font-weight: 400;font-style: normal;">// 你的业务service</span><br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(117, 113, 94);background: rgba(0, 0, 0, 0);display: inline;width: 577px;text-decoration: none solid rgb(117, 113, 94);font-weight: 400;font-style: normal;">/**<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> * 查询 通过查询缓存是否存在驱动缓存加载 建议在前置业务保证id对应数据是绝对存在于数据库中的<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> */</span><br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(248, 248, 242);background: rgba(0, 0, 0, 0);display: inline;width: 159px;text-decoration: none solid rgb(248, 248, 242);font-weight: 400;font-style: normal;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 40px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">public</span> Xx <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 12px;color: rgb(166, 226, 46);background: rgba(0, 0, 0, 0);display: inline;width: 33px;text-decoration: none solid rgb(166, 226, 46);font-weight: 400;font-style: normal;">getXx</span>(<span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 12px;color: rgb(248, 248, 242);background: rgba(0, 0, 0, 0);display: inline;width: 39px;text-decoration: none solid rgb(248, 248, 242);font-weight: 400;font-style: normal;"><span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-size: 12px;color: rgb(249, 38, 114);background: rgba(0, 0, 0, 0);display: inline;width: 20px;text-decoration: none solid rgb(249, 38, 114);font-weight: 400;font-style: normal;">int</span> id</span>) </span>{<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;font-size: 12px;color: rgb(117, 113, 94);background: rgba(0, 0, 0, 0);display: inline;width: 165px;text-decoration: none solid rgb(117, 113, 94);font-weight: 400;font-style: normal;">// 1.查询缓存里面有没有数据</span><br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> Xx xxCache = getXxFromCache(id);<br mpa-from-tpl="t" style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="margin: 0px;padding: 0px;outline: 0px;max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;font-family: CourierNewPS-ItalicMT;fo
作者：微信小助手
<section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px;color: black;padding-right: 10px;padding-left: 10px;line-height: 1.6;letter-spacing: 0px;word-break: break-word;text-align: left;font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;" data-mpa-powered-by="yiban.io"> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">前言</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">大家好，我是捡田螺的小男孩。作为后端开发，不管是什么语言，<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">Java</code>、<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">Go</code>还是<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">C++</code>，其背后的后端思想都是类似的。后面打算出一个后端思想的技术专栏，主要包括后端的一些设计、或者后端规范相关的，希望对大家日常工作有帮助哈。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">我们做后端开发工程师，主要工作就是：<strong>如何把一个接口设计好</strong>。所以，今天就给大家介绍，设计好接口的36个锦囊。本文就是后端思想专栏的第一篇哈。</p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img" data-galleryid="" data-ratio="0.8443782576321668" data-s="300,640" src="/upload/bb1999c0ff94b2739d4dcc3a8fad15f5.png" data-type="png" data-w="1343" style=""></p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">1. 接口参数校验</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span><br></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">入参出参校验是每个程序员必备的基本素养。你设计的接口，必须先校验参数。比如入参是否允许为空，入参长度是否符合你的预期长度。这个要养成习惯哈，日常开发中，很多低级bug都是不校验参数导致的。</p> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">比如你的数据库表字段设置为<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">varchar(16)</code>,对方传了一个32位的字符串过来，如果你不校验参数，<strong>插入数据库直接异常了</strong>。</p> </blockquote> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">出参也是，比如你定义的接口报文，参数是不为空的，但是你的接口返回参数，没有做校验，因为程序某些原因，直返回别人一个<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">null</code>值。。。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7933333333333333" src="/upload/f5eb5e34b613e6a48fb80964033dcbb.png" data-type="gif" data-w="150" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">2. 修改老接口时，注意接口的兼容性</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">很多bug都是因为修改了对外旧接口，但是却<strong>不做兼容</strong>导致的。关键这个问题多数是比较严重的，可能直接导致系统发版失败的。新手程序员很容易犯这个错误哦~</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7916666666666666" src="/upload/24b55a243cb9f99eb42faba698a68f76.png" data-type="png" data-w="240" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">所以，如果你的需求是在原来接口上修改，尤其这个接口是对外提供服务的话，一定要考虑接口兼容。举个例子吧，比如dubbo接口，原本是只接收A，B参数，现在你加了一个参数C，就可以考虑这样处理：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/eytJa9K5jkoibl85XeZqwTVsIibSZveTJ8gS6ibJxMaThWxnDpVmB84vdicaBJCMFKAs6Yb7XJPoia8tRTucD1aTDpU59bKl4ZWts/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(39, 40, 34);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #272822;border-radius: 5px;">//老接口<br>void oldService(A,B){<br> //兼容新接口，传个null代替C<br> newService(A,B,null);<br>}<br><br>//新接口，暂时不能删掉老接口，需要做兼容。<br>void newService(A,B,C){<br> ...<br>}<br></code></pre> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">3. 设计接口时，充分考虑接口的可扩展性</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">要根据实际业务场景设计接口，充分考虑接口的可扩展性。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">比如你接到一个需求：是用户添加或者修改员工时，需要刷脸。那你是反手提供一个员工管理的提交刷脸信息接口？还是先思考：提交刷脸是不是通用流程呢？比如转账或者一键贴现需要接入刷脸的话，你是否需要重新实现一个接口呢？还是当前按业务类型划分模块，复用这个接口就好，保留接口的可扩展性。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">如果按模块划分的话，未来如果其他场景比如一键贴现接入刷脸的话，不用再搞一套新的接口，只需要新增枚举，然后复用刷脸通过流程接口，实现一键贴现刷脸的差异化即可。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.4934696195343555" src="/upload/7356cae98194b7144574f34f8085f971.png" data-type="png" data-w="1761" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">4.接口考虑是否需要防重处理</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">如果前端重复请求，你的逻辑如何处理？是不是考虑接口去重处理。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">当然，如果是查询类的请求，其实不用防重。如果是更新修改类的话，尤其金融转账类的，就要过滤重复请求了。简单点，你可以使用Redis防重复请求，同样的请求方，一定时间间隔内的相同请求，考虑是否过滤。当然，转账类接口，并发不高的话，<strong>推荐使用数据库防重表</strong>，以<strong>唯一流水号作为主键或者唯一索引</strong>。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.37011173184357543" src="/upload/fe249d02775278d055dbdb5cf3ca46ea.png" data-type="png" data-w="716" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">5. 重点接口，考虑线程池隔离。</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">一些登陆、转账交易、下单等重要接口，考虑线程池隔离哈。如果你所有业务都共用一个线程池，有些业务出bug导致线程池阻塞打满的话，那就杯具了，<strong>所有业务都影响了</strong>。因此进行线程池隔离，重要业务分配多一点的核心线程，就更好保护重要业务。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.3845454545454545" src="/upload/fb968bfc1ff617b0686f786216de5ea.png" data-type="png" data-w="1100" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">6. 调用第三方接口要考虑异常和超时处理</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">如果你调用第三方接口，或者分布式远程服务的的话，需要考虑：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 异常处理 </section></li> </ul> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">比如，你调别人的接口，如果异常了，怎么处理，是重试还是当做失败还是告警处理。</p> </blockquote> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 接口超时 </section></li> </ul> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">没法预估对方接口一般多久返回，一般设置个超时断开时间，以保护你的接口。<strong>之前见过一个生产问题</strong>，就是http调用不设置超时时间，最后响应方进程假死，请求一直占着线程不释放，拖垮线程池。</p> </blockquote> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 重试次数 </section></li> </ul> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">你的接口调失败，需不需要重试？重试几次？需要站在业务上角度思考这个问题</p> </blockquote> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.6174496644295302" src="/upload/d75d85757e090d3c2835f9e4ad17db6b.png" data-type="gif" data-w="298" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">7. 接口实现考虑熔断和降级</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">当前互联网系统一般都是分布式部署的。而分布式系统中经常会出现某个基础服务不可用，最终导致整个系统不可用的情况, 这种现象被称为<strong>服务雪崩效应</strong>。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">比如分布式调用链路<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">A->B->C....</code>，下图所示：</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.5128983308042488" src="/upload/101c845d854be8ce78cbbcea6f66a33f.png" data-type="png" data-w="659" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">如果服务C出现问题，比如是<strong>因为慢SQL导致调用缓慢</strong>，那将导致B也会延迟，从而A也会延迟。堵住的A请求会消耗占用系统的线程、IO等资源。当请求A的服务越来越多，占用计算机的资源也越来越多，最终会导致系统瓶颈出现，造成其他的请求同样不可用，最后导致业务系统崩溃。</p> </blockquote> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">为了应对服务雪崩, 常见的做法是<strong>熔断和降级</strong>。最简单是加开关控制，当下游系统出问题时，开关降级，不再调用下游系统。还可以选用开源组件<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">Hystrix</code>。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">8. 日志打印好，接口的关键代码，要有日志保驾护航。</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">关键业务代码无论身处何地，都应该有足够的日志保驾护航。比如：你实现转账业务，转个几百万，然后转失败了，接着客户投诉，然后你还没有打印到日志，想想那种水深火热的困境下，你却毫无办法。。。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">那么，你的转账业务都需要哪些日志信息呢？至少，方法调用前，入参需要打印需要吧，接口调用后，需要捕获一下异常吧，同时打印异常相关日志吧，如下：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/eytJa9K5jkoibl85XeZqwTVsIibSZveTJ8gS6ibJxMaThWxnDpVmB84vdicaBJCMFKAs6Yb7XJPoia8tRTucD1aTDpU59bKl4ZWts/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(39, 40, 34);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #272822;border-radius: 5px;">public void transfer(TransferDTO transferDTO){<br> log.info(<span style="color: #a6e22e;line-height: 26px;">"invoke tranfer begin"</span>);<br> //打印入参<br> log.info(<span style="color: #a6e22e;line-height: 26px;">"invoke tranfer,paramters:{}"</span>,transferDTO);<br> try {<br> res= transferService.transfer(transferDTO);<br> }catch(Exception e){<br> log.error(<span style="color: #a6e22e;line-height: 26px;">"transfer fail,account：{}"</span>,<br> transferDTO.getAccount（）)<br> log.error(<span style="color: #a6e22e;line-height: 26px;">"transfer fail,exception:{}"</span>,e);<br> }<br> log.info(<span style="color: #a6e22e;line-height: 26px;">"invoke tranfer end"</span>);<br> }<br></code></pre> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">之前写过一篇打印日志的15个建议，大家可以看看哈：<a href="https://mp.weixin.qq.com/s?__biz=Mzg3NzU5NTIwNg==&mid=2247494838&idx=1&sn=cdb15fd346bddf3f8c1c99f0efbd67d8&chksm=cf22339ff855ba891616c79d4f4855e228e34a9fb45088d7acbe421ad511b8d090a90f5b019f&token=162724582&lang=zh_CN&scene=21#wechat_redirect" style="font-weight: bold;color: rgb(239, 112, 96);border-bottom: 1px solid rgb(239, 112, 96);" data-linktype="2">工作总结！日志打印的15个建议</a></p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">9. 接口的功能定义要具备单一性</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">单一性是指接口做的事情比较单一、专一。比如一个登陆接口，它做的事情就只是校验账户名密码，然后返回登陆成功以及<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">userId</code>即可。<strong>但是如果你为了减少接口交互，把一些注册、一些配置查询等全放到登陆接口，就不太妥。</strong></p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">其实这也是微服务一些思想，接口的功能单一、明确。比如订单服务、积分、商品信息相关的接口都是划分开的。将来拆分微服务的话，是不是就比较简便啦。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">10.接口有些场景，使用异步更合理</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">举个简单的例子，比如你实现一个用户注册的接口。用户注册成功时，发个邮件或者短信去通知用户。这个邮件或者发短信，就更适合异步处理。因为总不能一个通知类的失败，导致注册失败吧。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">至于做异步的方式，简单的就是<strong>用线程池</strong>。还可以使用消息队列，就是用户注册成功后，生产者产生一个注册成功的消息，消费者拉到注册成功的消息，就发送通知。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.2038327526132404" src="/upload/a4c7ec8168971466a614165510f968bc.png" data-type="png" data-w="1148" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">不是所有的接口都适合设计为同步接口。比如你要做一个转账的功能，如果你是单笔的转账，你是可以把接口设计同步。用户发起转账时，客户端在静静等待转账结果就好。如果你是批量转账，一个批次一千笔，甚至一万笔的，你则可以把接口设计为异步。就是用户发起批量转账时，持久化成功就先返回受理成功。然后用户隔十分钟或者十五分钟等再来查转账结果就好。又或者，批量转账成功后，再回调上游系统。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.44280442804428044" src="/upload/323f5de24c71a6d874868fed8494a536.png" data-type="png" data-w="813" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">11. 优化接口耗时，远程串行考虑改并行调用</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">假设我们设计一个APP首页的接口，它需要查用户信息、需要查banner信息、需要查弹窗信息等等。那你是一个一个接口串行调，还是并行调用呢？</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.2056239015817223" src="/upload/bd732ac66e75dff59af692be8fc8c7e.png" data-type="png" data-w="1138" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">如果是串行一个一个查，比如查用户信息200ms，查banner信息100ms、查弹窗信息50ms，那一共就耗时<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">350ms</code>了，如果还查其他信息，那耗时就更大了。这种场景是可以改为并行调用的。也就是说查用户信息、查banner信息、查弹窗信息，可以同时发起。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.5670731707317073" src="/upload/ebb51ddb2e8841320b2d0e08d7dd51a7.png" data-type="png" data-w="820" style="display: block;margin-right: auto;margin-left: auto;"> </figure> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">在Java中有个异步编程利器：<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">CompletableFuture</code>，就可以很好实现这个功能。有兴趣的小伙伴可以看我之前这个文章哈：<a href="https://mp.weixin.qq.com/s?__biz=Mzg3NzU5NTIwNg==&mid=2247490456&idx=1&sn=95836324db57673a4d7aea4fb233c0d2&chksm=cf21c4b1f8564da72dc7b39279362bcf965b1374540f3b339413d138599f7de59a5f977e3b0e&token=1260947715&lang=zh_CN&scene=21#wechat_redirect" style="font-weight: bold;color: rgb(239, 112, 96);border-bottom: 1px solid rgb(239, 112, 96);" data-linktype="2">CompletableFuture详解</a></p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">12. 接口合并或者说考虑批量处理思想</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">数据库操作或或者是远程调用时，能批量操作就不要for循环调用。<img class="rich_pages wxw-img" data-ratio="1.0433333333333332" src="/upload/ef43e5682ac7f38bdd3f84c1bceeac22.png" data-type="png" data-w="300" style="display: block;margin-right: auto;margin-left: auto;"></p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">一个简单例子，我们平时一个列表明细数据插入数据库时，不要在for循环一条一条插入，建议一个批次几百条，进行批量插入。同理远程调用也类似想法，比如你查询营销标签是否命中，可以一个标签一个标签去查，也可以批量标签去查，那批量进行，效率就更高嘛。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/eytJa9K5jkoibl85XeZqwTVsIibSZveTJ8gS6ibJxMaThWxnDpVmB84vdicaBJCMFKAs6Yb7XJPoia8tRTucD1aTDpU59bKl4ZWts/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(39, 40, 34);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #272822;border-radius: 5px;">//反例<br><span style="color: #f92672;font-weight: bold;line-height: 26px;">for</span>(int i=0;i<n;i++){<br> remoteSingleQuery(param)<br>}<br><br>//正例<br>remoteBatchQuery(param);<br></code></pre> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">小伙伴们是否了解过<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">kafka</code>为什么这么快呢？其实其中一点原因，就是kafka<strong>使用批量消息</strong>提升服务端处理能力。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">13. 接口实现过程中，恰当使用缓存</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">哪些场景适合使用缓存？<strong>读多写少且数据时效要求越低的场景</strong>。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">缓存用得好，可以承载更多的请求，提升查询效率，减少数据库的压力。</p> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">比如一些平时变动很小或者说几乎不会变的商品信息，可以放到缓存，请求过来时，先查询缓存，如果没有再查数据库，并且把数据库的数据更新到缓存。但是，使用缓存增加了需要考虑这些点：缓存和数据库一致性如何保证、集群、缓存击穿、缓存雪崩、缓存穿透等问题。</p> </blockquote> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 保证数据库和缓存一致性： <strong style="color: black;">缓存延时双删、删除缓存重试机制、读取biglog异步删除缓存</strong> </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 缓存击穿：设置数据永不过期 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 缓存 <span style="color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;font-size: 16px;text-align: left;background-color: rgb(255, 249, 249);">雪崩</span>：Redis集群高可用、均匀设置过期时间 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 缓存穿透：接口层校验、查询为空设置个默认空值标记、布隆过滤器。 </section></li> </ul> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">一般用<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">Redis</code>分布式缓存，当然有些时候也可以考虑使用本地缓存，如<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">Guava Cache、Caffeine</code>等。使用本地缓存有些缺点，就是无法进行大数据存储，并且应用进程的重启，缓存会失效。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">14. 接口考虑热点数据隔离性</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">瞬时间的高并发，可能会打垮你的系统。可以做一些热点数据的隔离。比如<strong>业务隔离、系统隔离、用户隔离、数据隔离</strong>等。</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 业务隔离性，比如12306的分时段售票，将热点数据分散处理，降低系统负载压力。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 系统隔离：比如把系统分成了用户、商品、社区三个板块。这三个块分别使用不同的域名、服务器和数据库，做到从接入层到应用层再到数据层三层完全隔离。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 用户隔离：重点用户请求到配置更好的机器。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 数据隔离：使用单独的缓存集群或者数据库服务热点数据。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">15. 可变参数配置化，比如红包皮肤切换等</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">假如产品经理提了个红包需求，圣诞节的时候，红包皮肤为圣诞节相关的，春节的时候，为春节红包皮肤等。</p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">如果在代码写死控制，可有类似以下代码：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;
作者：微信小助手
<section style="font-size: 15px;line-height: 1.6;box-sizing: border-box;font-style: normal;font-weight: 400;text-align: justify;" data-mpa-powered-by="yiban.io"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgba(0, 0, 0, 0.5);font-size: 14px;text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">vivo 互联网服务器团队-YuanPeng</p> </section> </section> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">一、概述</p> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">从容器技术的推广以及 Kubernetes成为容器调度管理领域的事实标准开始，云原生的理念和技术架构体系逐渐在生产环境中得到了越来越广泛的应用实践。在云原生的体系下，面对高度的弹性、动态的应用生命周期管理以及微服务化等特点，传统的监控体系已经难以应对和支撑，因此新一代云原生监控体系应运而生。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">当前，以Prometheus为核心的监控系统已成为云原生监控领域的事实标准。Prometheus作为新一代云原生监控系统，拥有强大的查询能力、便捷的操作、高效的存储以及便捷的配置操作等特点，但任何一个系统都不是万能的，面对复杂多样的生产环境，单一的Prometheus系统也无法满足生产环境的各种监控需求，都需要根据环境的特点来构建适合的监控方法和体系。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">本文以vivo容器集群监控实践经验为基础，探讨了云原生监控体系架构如何构建、遇到的挑战以及相应的对策。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">二、云原生监控体系</p> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">2.1 云原生监控的特征和价值</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">云原生监控相比于传统监控，有其特征和价值，可归纳为下表：</p> </section> <section style="min-height: 40px;margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="width: 100%;margin: 0px auto -10px;box-sizing: border-box;"> <table width="100%"> <tbody style="box-sizing: border-box;"> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:0" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:0.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(65, 95, 255);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;color: rgb(255, 255, 255);box-sizing: border-box;"> <p style="text-align: center;white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">特征</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:0.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(65, 95, 255);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;color: rgb(255, 255, 255);box-sizing: border-box;"> <p style="text-align: center;white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">价值</p> </section> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:1" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:1.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">监控系统以云原生方式部署</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:1.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;box-sizing: border-box;"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">标准化部署和升级</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">统一的编排调度</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">弹性伸缩</p></li> </ul> </section> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:2" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:2.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">统一的云原生监控标准</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:2.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">标准的监控接口<br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">标准的监控数据格式</p></li> </ul> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:3" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:3.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">采集端对业务侵入性极小 </p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:3.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">云原生应用自带监控接口</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">传统应用通过exporter提供监控数据</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">应用接入监控系统的复杂度低</p></li> </ul> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:4" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:4.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">云原生一体化的设计</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:4.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">适用于容器动态生命周期的特点</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">支撑容器量级的海量监控数据</p></li> </ul> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:12.classicTable1:5" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:5.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">完整的社区生态</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:12.classicTable1:5.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="59.9900%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">丰富的开源项目+持续的演进</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">强大的社区支持</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">全球顶尖厂商的丰富生产实践经验</p></li> </ul> </section></td> </tr> </tbody> </table> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">2.2 云原生监控生态简介</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">CNCF生态全景图中监控相关的项目如下图（参考https://landscape.cncf.io/），下面重点介绍几个项目：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">Prometheus（已毕业）</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus是一个强大的监控系统，同时也是一个高效的时序数据库，并且具有完整的围绕它为核心的监控体系解决方案。单台Prometheus就能够高效的处理大量监控数据，并且具备非常友好且强大的PromQL语法，可以用来灵活查询各种监控数据以及告警规则配置。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus是继Kubernetes之后的第二个CNCF “毕业”项目（也是目前监控方向唯一“毕业”的项目），开源社区活跃，在Github上拥有近4万Stars。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus的Pull指标采集方式被广泛采用，很多应用都直接实现了基于Prometheus采集标准的metric接口来暴露自身监控指标。即使是没有实现metric接口的应用，大部分在社区里都能找到相应的exporter来间接暴露监控指标。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">但Prometheus仍然存在一些不足，比如只支持单机部署，Prometheus自带时序库使用的是本地存储，因此存储空间受限于单机磁盘容量，在大数据量存储的情况下，prometheus的历史数据查询性能会有严重瓶颈。因此在大规模生产场景下，单一prometheus难以存储长期历史数据且不具备高可用能力。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">Cortex（孵化中）</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Cortex对Prometheus进行了扩展，提供多租户方式，并为Prometheus提供了对接持久化存储的能力，支持Prometheus实例水平扩展，以及提供多个Prometheus数据的统一查询入口。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">Thanos（孵化中）</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Thanos通过将Prometheus监控数据存储到对象存储，提供了一种长期历史监控数据存储的低成本解决方案。Thanos通过Querier组件给Prometheus集群提供了全局视图（全局查询），并能将Prometheus的样本数据压缩机制应用到对象存储的历史数据中，还能通过降采样功能提升大范围历史数据的查询速度，且不会引起明显的精度损失。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">Grafana</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Grafana是一个开源的度量分析与可视化套件，主要在监控领域用于时序数据的图标自定义和展示，UI非常灵活，有丰富的插件和强大的扩展能力，支持多种不同的数据源（Graphite, InfluxDB, OpenTSDB, Prometheus, Elasticsearch, Druid等等）。Grafana还提供可视化的告警定制能力，能够持续的评估告警指标，发送告警通知。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">此外，Grafana社区提供了大量常用系统/组件的监控告警面板配置，可以一键在线下载配置，简单便捷。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">VictoriaMetrics</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">VictoriaMetrics是一个高性能、经济且可扩展的监控解决方案和时序数据库，可以作为Prometheus的长期远程存储方案，支持PromQL查询，并与Grafana兼容，可用于替换Prometheus作为Grafana的数据源。具有安装配置简单、低内存占用、高压缩比、高性能以及支持水平扩展等特性。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">AlertManager</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">AlertManager是一个告警组件，接收Prometheus发来的告警，通过分组、沉默、抑制等策略处理后，通过路由发送给指定的告警接收端。告警可以按照不同的规则发送给不同的接收方，支持多种常见的告警接收端，比如Email，Slack，或通过webhook方式接入企业微信、钉钉等国内IM工具。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.5595744680851064" data-s="300,640" src="/upload/2fb0b3b65ac39cc4d64408a1e9c8f55d.png" data-type="png" data-w="940" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">2.3 如何搭建一套简单的云原生监控系统</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">上文了解了云原生监控领域的常用工具后，该如何搭建一套简单的云原生监控系统？下图给出了Prometheus社区官方提供的方案：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;text-align: center;"><br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.38828125" data-s="300,640" src="/upload/bb8d42422f49020b84b8e16f4ab99e73.jpg" data-type="jpeg" data-w="1280" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;text-align: center;"><span style="color: rgb(136, 136, 136);font-size: 14px;"><br></span></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;text-align: center;"><span style="color: rgb(136, 136, 136);font-size: 14px;">（出处：Prometheus社区）</span></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">上述系统展开阐述如下：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">所有监控组件都是以云原生的方式部署，即容器化部署、用Kubernetes来统一管理。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus负责指标采集和监控数据存储，并可以通过文件配置或Kubernetes服务发现方式来自动发现采集目标。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">应用可以通过自身的Metric接口或相应的exporter来让Prometheus拉取监控数据。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">一些短暂的自定义采集指标，可以通过脚本程序采集并推送给Pushgateway组件，来让Prometheus拉取。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus配置好告警规则，将告警数据发送给Alertmanager，由Alertmanager按照一定规则策略处理后路由给告警接收方。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Grafana配置Prometheus作为数据源，通过PromQL查询监控数据后，做告警面板展示。</p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">2.4 如何构建能力开放、稳定高效的云原生监控体系</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">上文展示了社区官方给出的监控系统搭建方案，但该方案在生产环境应用时存在的主要问题：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus单机无法存储大量长期历史数据；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">不具备高可用能力；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">不具备横向扩展能力；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">缺少多维度的监控统计分析能力。</p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">那么对于大规模复杂生产环境，如何构建能力开放、稳定高效的云原生监控体系？</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">综合vivo自身容器集群监控实践经验、各类云原生监控相关文档以及技术大会上各家厂商的技术架构分享，可以总结出适合大规模生产场景的云原生监控体系架构，下图展示了体系架构的分层模型。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">通过云原生方式部署，底层使用Kubernetes作为统一的控制管理平面。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">监控层采用Prometheus集群作为采集方案，Prometheus集群通过开源/自研高可用方案来保证无单点故障以及提供负载均衡能力，监控指标则通过应用/组件的自身Metric API或exporter来暴露。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">告警数据发给告警组件按照指定规则进行处理，再由webhook转发给公司的告警中心或其他通知渠道。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">数据存储层，采用高可用可扩展的时序数据库方案来存储长期监控数据，同时也用合适的数仓系统存储一份来做更多维度的监控数据统计分析，为上层做数据分析提供基础。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">数据分析处理层，可以对监控数据做进一步的分析处理，提供更多维度的报表，挖掘更多有价值的信息，甚至可以利用机器学习等技术实现故障预测、故障自愈等自动化运维目的。</p></li> </ul> <p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.5875" data-s="300,640" src="/upload/3f29b597dd5a10e2545b4ae397253b06.jpg" data-type="jpeg" data-w="1280" style=""></p> <p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">三、vivo容器集群监控架构</p> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">任何系统的架构设计一定是针对生产环境和业务需求的特点，以满足业务需求和高可用为前提，在综合考虑技术难度、研发投入和运维成本等综合因素后，设计出最符合当前场景的架构方案。因此，在详解vivo容器集群监控架构设计之前，需要先介绍下背景：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">生产环境</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">vivo目前有多个容器化生产集群，分布在若干机房，目前单集群最大规模在1000~2000节点。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">监控需求</strong><br style="box-sizing: border-box;"></p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">需要满足生产高可用，监控范围主要包括容器集群指标、物理机运行指标和容器（业务）指标，其中业务监控告警还需要通过公司的基础监控平台来展示和配置。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">告警需求</strong><br style="box-sizing: border-box;"></p></li> </ul> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">告警需要可视化的配置方式，需要发送给公司的告警中心，并有分级分组等策略规则需求。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">数据分析需求</strong><br style="box-sizing: border-box;"></p></li> </ul> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">有各类丰富的周、月度、季度统计报表需求。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">3.1 监控高可用架构设计</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">结合上文说明的环境和需求特点，vivo当前监控架构的设计思路：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">每个生产集群都有独立的监控节点用于部署监控组件，Prometheus按照采集目标服务划分为多组，每组Prometheus都是双副本部署保证高可用。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">数据存储采用VictoriaMetrics，每个机房部署一套VictoriaMetrics集群，同一机房内集群的Prometheus均将监控数据remote-write到VM中，VM配置为多副本存储，保证存储高可用。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Grafana对接Mysql集群，Grafana自身做到无状态，实现Grafana多副本部署。Grafana使用VictoriaMetrics作为数据源。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">通过拨测监控实现Prometheus自身的监控告警，在Prometheus异常时能及时收到告警信息。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">集群层面的告警使用Grafana的可视化告警配置，通过自研webhook将告警转发给公司告警中心，自研webhook还实现了分级分组等告警处理策略。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">容器层面（业务）的监控数据则通过自研Adapter转发给Kafka，进而存储到公司基础监控做业务监控展示和告警配置，同时也存储一份到Druid做更多维度的统计报表。</p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.525" data-s="300,640" src="/upload/c150cfc036285667fb2b5f0a4974273d.jpg" data-type="jpeg" data-w="1280" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">前文介绍了社区的Cortex和Thanos高可用监控方案，这两个方案在业界均有生产级的实践经验，但为什么我们选择用Prometheus双副本+VictoriaMetrics的方案？</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">主要原因有以下几点：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Cortex在网上能找到的相关实践文档较少。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Thanos需要使用对象存储，实际部署时发现Thanos的配置比较复杂，参数调优可能比较困难，另外Thanos需要在Prometheus Pod里部署其SideCar组件，而我们Prometheus部署采用的是Operator自动部署方式，嵌入SideCar比较麻烦。另外，在实测中对Thanos组件进行监控时发现，Thanos因为Compact和传输Prometheus数据存储文件等原因，时常出现CPU和网络的尖峰。综合考虑后认为Thanos的后期维护成本较高，因此没有采用。</p><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">VictoriaMetrics部署配置比较简单，有很高的存储、查询和压缩性能，支持数据去重，不依赖外部系统，只需要通过Prometheus配置remote-write写入监控数据即可，并且与Grafana完全兼容。既满足我们长期历史数据存储和高可用需求，同时维护成本很低。从我们对VictoriaMetrics自身组件的监控观察来看，运行数据平稳，并且自生产使用以来，一直稳定运行无故障。</p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="font-size: 16px;color: rgb(65, 94, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">3.2 监控数据转发层组件高可用设计</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">由于Prometheus采用双副本部署高可用方案，数据存储如何去重是需要设计时就考虑的。VictoriaMetrics本身支持存储时去重，因此VictoriaMetrics这一侧的数据去重得到天然解决。但监控数据通过Kafka转发给基础监控平台和OLAP这一侧的去重该如何实现？</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">我们设计的方案，如下图，是通过自研Adapter “分组选举”方式实现去重。即每个Prometheus副本对应一组Adapter，两组Adapter之间会进行选主，只有选举为Leader的那组Adapter才会转发数据。通过这种方式既实现了去重，也借用K8s service来支持Adapter的负载均衡。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">此外，Adapter具备感知Prometheus故障的能力，当Leader Prometheus发生故障时，Leader Adapter会感知到并自动放弃Leader身份，从而切换到另一组Adapter继续传输数据，确保了“双副本高可用+去重”方案的有效性。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.53515625" data-s="300,640" src="/upload/e445495aaf3be9b0258244c9ec61a9e0.jpg" data-type="jpeg" data-w="1280" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">四、 容器监控实践的挑战和对策</p> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">我们在容器集群监控实践的过程中，遇到的一些困难和挑战，总结如下：</p> </section> <section style="min-height: 40px;margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="width: 100%;margin: 0px auto -10px;box-sizing: border-box;"> <table width="100%"> <tbody style="box-sizing: border-box;"> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:0" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:0.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(65, 95, 255);box-sizing: border-box;padding: 0px;" width="20.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;color: rgb(255, 255, 255);box-sizing: border-box;"> <p style="text-align: center;white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">问题</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:0.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(65, 95, 255);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;color: rgb(255, 255, 255);box-sizing: border-box;"> <p style="text-align: center;white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">挑战点</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:0.td@@2" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(65, 95, 255);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="color: rgb(255, 255, 255);box-sizing: border-box;" powered-by="xiumi.us"> <p style="text-align: center;white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">对策</p> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:1" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="3" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:1.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="20.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">大规模性能问题</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:1.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="margin: 10px 0%;box-sizing: border-box;" powered-by="xiumi.us"> <section style="line-height: 1.8;letter-spacing: 0px;box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus目前人工分组分片，实例的负载是不均衡的</p> </section> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:1.td@@2" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">社区有开源项目支持自动分片和负载均衡</p> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:2" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:2.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus在压力大时会出现丢弃少量数据现象，影响OLAP端分析监控数据的准确性</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:2.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">需要负载均衡能力，降低Prometheus实例的压力</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">降低OLAP端采集精度，降低丢数据的影响</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">升级Prometheus版本，保障更好的性能</p></li> </ul> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:3" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:3.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">时序数据库性能和扩容</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:3.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">VictoriaMetrics吞吐和容量支持扩容</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">梳理精简监控指标，对无用指标进行过滤</p></li> </ul> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:4" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="2" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:4.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="20.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">云原生监控体系落地</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:4.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;">与公司已有监控体系的融合<br style="box-sizing: border-box;"></p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:4.td@@2" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">公司监控体系兼容云原生监控采集端和数据源格式</p> </section></td> </tr> <tr opera-tn-ra-comp="_$.pages:0.layers:0.comps:89.classicTable1:5" style="box-sizing: border-box;" powered-by="xiumi.us"> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:5.td@@0" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">业务全面容器化后，更丰富的监控维度建设</p> </section></td> <td colspan="1" rowspan="1" opera-tn-ra-cell="_$.pages:0.layers:0.comps:89.classicTable1:5.td@@1" style="border-width: 1px;border-color: rgb(255, 255, 255);border-style: solid;background-color: rgb(245, 245, 245);box-sizing: border-box;padding: 0px;" width="40.0000%"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">持续跟进云原生监控生态的发展</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">帮助业务团队了解容器监控原理架构，监控面板定制方法，以及自定义exporter的开发方法</p></li> </ul> </section></td> </tr> </tbody> </table> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">五、未来展望</p> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">监控的目标是为了更高效可靠的运维，准确及时的发现问题。更高的目标是基于监控实现自动化的运维，甚至是智能运维（AIOPS）。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">基于目前对容器集群监控的经验总结，未来在监控架构上可以做的提升点包括：</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Prometheus自动化分片及采集Target自动负载均衡；<br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">AI预测分析潜在故障；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">故障自愈；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">通过数据分析设定合适的告警阈值；</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">优化告警管控策略。</p></li> </ul> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">没有一种架构设计是一劳永逸的，必须要随着生产环境和需求的变化，以及技术的发展来持续演进。我们在云原生监控这条路上，需要继续不忘初心，砥砺前行。</p> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;vertical-align: top;width: 20%;box-sizing: border-box;"> <section powered-by="xiumi.us"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> <p style="margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> </section> </section> </section>
作者：微信小助手
<section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px;color: black;padding: 0 10px;line-height: 1.6;word-spacing: 0px;letter-spacing: 0px;word-wrap: break-word;text-align: left;word-break: break-all;font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, 'PingFang SC', Cambria, Cochin, Georgia, Times, 'Times New Roman', serif;"> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">大家好，我是三鸽，这期继续更新<a href="https://mp.weixin.qq.com/mp/appmsgalbum?__biz=MzkwODE5ODM0Ng==&action=getalbum&album_id=2041709347461709827&scene=173&from_msgid=2247489569&from_itemidx=1&count=3&nolastread=1#wechat_redirect" style="text-decoration: none;word-wrap: break-word;border-bottom: 1px solid #1e6bb8;color: hsl(187, 100%, 45%);font-weight: normal;border-bottom-color: hsl(187, 100%, 45%);" data-linktype="2">面渣逆袭</a>系列，主角是MySQL。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">不知不觉，面渣逆袭系列已经肝了差不多十篇，每一篇都是上万字，几十图，基本上涵盖了面试的主要知识点，这期MySQL结束之后，这个系列可能会暂时告一段落，作为面渣逆袭系列第一阶段的收官之作，大家多多<strong>点赞</strong>、<strong>收藏</strong>哦！</p> <section class="mp_profile_iframe_wrp"> <mpprofile class="js_uneditable custom_select_card mp_profile_iframe" data-pluginname="mpprofile" data-id="MzkwODE5ODM0Ng==" data-headimg="http://mmbiz.qpic.cn/mmbiz_png/PMZOEonJxWdFLJg0sAOqwHB1mb24icMADUgxm1qZQft5aN3H37NAmQnOvpGB7J9JVHxC6NSiacxbBP1DYdhIAeyA/0?wx_fmt=png" data-nickname="三分恶" data-alias="Fighter3FullStack" data-signature="CSDN博客专家、优质创作者，华为云云享专家；肝过外包、混过国企，目前在一家跨境电商搬砖；写过诗，打过拳，佛系小码农。认真讲技术，随性侃人生，关注我，我们一起走的更远。" data-from="0"></mpprofile> </section> <h1 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;color: black;font-size: 1.7em;font-weight: normal;border-bottom: 2px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="background: hsl(216, 100%, 68%);color: white;padding: 3px 10px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">基础</span></h1> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.5143769968051118" src="/upload/cae002c6aeec2cd6b137f862b5d405eb.png" data-type="png" data-w="313" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> MySQ Logo </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">作为SQL Boy，基础部分不会有人不会吧？面试也不怎么问，基础掌握不错的小伙伴可以<strong style="font-weight: bold;color: hsl(216, 80%, 44%);">跳过</strong>这一部分。当然，可能会现场写一些SQL语句，SQ语句可以通过牛客、LeetCode、LintCode之类的网站来练习。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">1. 什么是内连接、外连接、交叉连接、笛卡尔积呢？</span></h2> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 内连接（inner join）：取得两张表中满足存在连接匹配关系的记录。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 外连接（outer join）：不只取得两张表中满足存在连接匹配关系的记录，还包括某张表（或两张表）中不满足匹配关系的记录。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 交叉连接（cross join）：显示两张表所有记录一一对应，没有匹配关系进行筛选，它是笛卡尔积在SQL中的实现，如果A表有m行，B表有n行，那么A和B交叉连接的结果就有m*n行。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 笛卡尔积：是数学中的一个概念，例如集合A={a,b}，集合B={1,2,3}，那么A✖️B={<a,o>,<a,1>,<a,2>,<b,0>,<b,1>,<b,2>,}。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">2. 那MySQL 的内连接、左连接、右连接有有什么区别？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">MySQL的连接主要分为内连接和外连接，外连接常用的有左连接、右连接。</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7867494824016563" src="/upload/b8a7e6edfe900ca8b84f353c1695fcd2.png" data-type="png" data-w="966" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> MySQL-joins-来源菜鸟教程 </figcaption> </figure> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> inner join 内连接，在两张表进行连接查询时，只保留两张表中完全匹配的结果集 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> left join 在两张表进行连接查询时，会返回左表所有的行，即使在右表中没有匹配的记录。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> right join 在两张表进行连接查询时，会返回右表所有的行，即使在左表中没有匹配的记录。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">3.说一下数据库的三大范式？</span></h2> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.608955223880597" src="/upload/e6481e7c7913f74949887140f5f9ec50.png" data-type="png" data-w="670" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> 数据库三范式 </figcaption> </figure> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 第一范式：数据表中的每一列（每个字段）都不可以再拆分。例如用户表，用户地址还可以拆分成国家、省份、市，这样才是符合第一范式的。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 第二范式：在第一范式的基础上，非主键列完全依赖于主键，而不能是依赖于主键的一部分。例如订单表里，存储了商品信息（商品价格、商品类型），那就需要把商品ID和订单ID作为联合主键，才满足第二范式。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 第三范式：在满足第二范式的基础上，表中的非主键只依赖于主键，而不依赖于其他非主键。例如订单表，就不能存储用户信息（姓名、地址）。 </section></li> </ul> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="1" src="/upload/d28ee12a8a622a52788119feb755854f.png" data-type="gif" data-w="300" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> 你设计遵守范式吗？ </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">三大范式的作用是为了控制数据库的冗余，是对空间的节省，实际上，一般互联网公司的设计都是反范式的，通过冗余一些数据，避免跨表跨库，利用空间换时间，提高性能。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">4.varchar与char的区别？</span></h2> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.27370689655172414" src="/upload/37550c667c359a1236921ea35ea9f2f6.png" data-type="png" data-w="928" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> varchar </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">char</strong>：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> char表示定长字符串，长度是固定的； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果插入数据的长度小于char的固定长度时，则用空格填充； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 因为长度固定，所以存取速度要比varchar快很多，甚至能快50%，但正因为其长度固定，所以会占据多余的空间，是空间换时间的做法； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 对于char来说，最多能存放的字符个数为255，和编码无关 </section></li> </ul> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">varchar</strong>：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> varchar表示可变长字符串，长度是可变的； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 插入的数据是多长，就按照多长来存储； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> varchar在存取方面与char相反，它存取慢，因为长度不固定，但正因如此，不占据多余的空间，是时间换空间的做法； </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 对于varchar来说，最多能存放的字符个数为65532 </section></li> </ul> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">日常的设计，对于长度相对固定的字符串，可以使用char，对于长度不确定的，使用varchar更合适一些。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">5.blob和text有什么区别？</span></h2> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> blob用于存储二进制数据，而text用于存储大字符串。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> blob没有字符集，text有一个字符集，并且根据字符集的校对规则对值进行排序和比较 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">6.DATETIME和TIMESTAMP的异同？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">相同点</strong>：</p> <ol data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: decimal;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 两个数据类型存储时间的表现格式一致。均为 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">YYYY-MM-DD HH:MM:SS</code> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 两个数据类型都包含「日期」和「时间」部分。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 两个数据类型都可以存储微秒的小数秒（秒后6位小数秒） </section></li> </ol> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">区别</strong>：</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7874564459930313" src="/upload/9ffcdf85c90bbc4e4138bfe315fba73c.png" data-type="png" data-w="574" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> DATETIME和TIMESTAMP的区别 </figcaption> </figure> <ol data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: decimal;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">日期范围</strong>：DATETIME 的日期范围是 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">1000-01-01 00:00:00.000000</code> 到 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">9999-12-31 23:59:59.999999</code>；TIMESTAMP 的时间范围是<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">1970-01-01 00:00:01.000000</code> UTC<code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);"> 到 ``2038-01-09 03:14:07.999999</code> UTC</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">存储空间</strong>：DATETIME 的存储空间为 8 字节；TIMESTAMP 的存储空间为 4 字节</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">时区相关</strong>：DATETIME 存储时间与时区无关；TIMESTAMP 存储时间与时区有关，显示的值也依赖于时区</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">默认值</strong>：DATETIME 的默认值为 null；TIMESTAMP 的字段默认不为空(not null)，默认值为当前时间(CURRENT_TIMESTAMP)</p> </section></li> </ol> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">7.MySQL中 in 和 exists 的区别？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">MySQL中的in语句是把外表和内表作hash 连接，而exists语句是对外表作loop循环，每次loop循环再对内表进行查询。我们可能认为exists比in语句的效率要高，这种说法其实是不准确的，要区分情景：</p> <ol data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: decimal;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果查询的两个表大小相当，那么用in和exists差别不大。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果两个表中一个较小，一个是大表，则子查询表大的用exists，子查询表小的用in。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> not in 和not exists：如果查询语句使用了not in，那么内外表都进行全表扫描，没有用到索引；而not extsts的子查询依然能用到表上的索引。所以无论那个表大，用not exists都比not in要快。 </section></li> </ol> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">8.MySQL里记录货币用什么字段类型比较好？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">货币在数据库中MySQL常用Decimal和Numric类型表示，这两种类型被MySQL实现为同样的类型。他们被用于保存与货币有关的数据。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">例如salary DECIMAL(9,2)，9(precision)代表将被用于存储值的总的小数位数，而2(scale)代表将被用于存储小数点后的位数。存储在salary列中的值的范围是从-9999999.99到9999999.99。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">DECIMAL和NUMERIC值作为字符串存储，而不是作为二进制浮点数，以便保存那些值的小数精度。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">之所以不使用float或者double的原因：因为float和double是以二进制存储的，所以有一定的误差。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">9.MySQL怎么存储emoji😊?</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">MySQL可以直接使用字符串存储emoji。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">但是需要注意的，utf8 编码是不行的，MySQL中的utf8是阉割版的 utf8，它最多只用 3 个字节存储字符，所以存储不了表情。那该怎么办？</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">需要使用utf8mb4编码。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/iahdQicCC5VBSsGkibhVicLUiaOMaR8fdicOEMC1UxVKS1icZrGaECGoK6NX4jHJ94R8iaJD1555Pj5rpbZK5b48mrZJOiccqakM9hjWB/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(40, 44, 52);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;"><span style="color: #c678dd;line-height: 26px;">alter</span> <span style="color: #c678dd;line-height: 26px;">table</span> blogs <span style="color: #c678dd;line-height: 26px;">modify</span> <span style="color: #c678dd;line-height: 26px;">content</span> <span style="color: #e6c07b;line-height: 26px;">text</span> <span style="color: #e6c07b;line-height: 26px;">CHARACTER</span> <span style="color: #c678dd;line-height: 26px;">SET</span> utf8mb4 <span style="color: #c678dd;line-height: 26px;">COLLATE</span> utf8mb4_unicode_ci <span style="color: #c678dd;line-height: 26px;">not</span> <span style="color: #56b6c2;line-height: 26px;">null</span>;<br></code></pre> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">10.drop、delete与truncate的区别？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">三者都表示删除，但是三者有一些差别：</p> <section data-tool="mdnice编辑器" style="overflow-x: auto;"> <table width="NaN"> <thead> <tr style="border-width: 1px 0px 0px;border-right-style: initial;border-bottom-style: initial;border-left-style: initial;border-right-color: initial;border-bottom-color: initial;border-left-color: initial;border-top-style: solid;border-top-color: rgb(204, 204, 204);background-color: white;"> <th style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;background-color: rgb(240, 240, 240);color: rgb(51, 51, 51);font-weight: normal;min-width: 85px;"><br></th> <th style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;background-color: rgb(240, 240, 240);color: rgb(51, 51, 51);font-weight: normal;min-width: 85px;">delete</th> <th style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;background-color: rgb(240, 240, 240);color: rgb(51, 51, 51);font-weight: normal;min-width: 85px;">truncate</th> <th style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;background-color: rgb(240, 240, 240);color: rgb(51, 51, 51);font-weight: normal;min-width: 85px;">drop</th> </tr> </thead> <tbody style="border-width: 0px;border-style: initial;border-color: initial;"> <tr style="border-width: 1px 0px 0px;border-right-style: initial;border-bottom-style: initial;border-left-style: initial;border-right-color: initial;border-bottom-color: initial;border-left-color: initial;border-top-style: solid;border-top-color: rgb(204, 204, 204);background-color: white;"> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">类型</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">属于DML</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">属于DDL</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">属于DDL</td> </tr> <tr style="border-width: 1px 0px 0px;border-right-style: initial;border-bottom-style: initial;border-left-style: initial;border-right-color: initial;border-bottom-color: initial;border-left-color: initial;border-top-style: solid;border-top-color: rgb(204, 204, 204);background-color: rgb(248, 248, 248);"> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">回滚</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">可回滚</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">不可回滚</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">不可回滚</td> </tr> <tr style="border-width: 1px 0px 0px;border-right-style: initial;border-bottom-style: initial;border-left-style: initial;border-right-color: initial;border-bottom-color: initial;border-left-color: initial;border-top-style: solid;border-top-color: rgb(204, 204, 204);background-color: white;"> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">删除内容</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">表结构还在，删除表的全部或者一部分数据行</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">表结构还在，删除表中的所有数据</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">从数据库中删除表，所有数据行，索引和权限也会被删除</td> </tr> <tr style="border-width: 1px 0px 0px;border-right-style: initial;border-bottom-style: initial;border-left-style: initial;border-right-color: initial;border-bottom-color: initial;border-left-color: initial;border-top-style: solid;border-top-color: rgb(204, 204, 204);background-color: rgb(248, 248, 248);"> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">删除速度</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">删除速度慢，需要逐行删除</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">删除速度快</td> <td style="font-size: 16px;border-width: 1px;border-style: solid;border-color: rgb(204, 204, 204);padding: 5px 10px;text-align: left;color: rgb(102, 102, 102);min-width: 85px;">删除速度最快</td> </tr> </tbody> </table> </section> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">因此，在不再需要一张表的时候，用drop；在想删除部分数据行时候，用delete；在保留表而删除所有数据的时候用truncate。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">11.UNION与UNION ALL的区别？</span></h2> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果使用UNION ALL，不会合并重复的记录行 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 效率 UNION 高于 UNION ALL </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">12.count(1)、count(*) 与 count(列名) 的区别？</span></h2> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.20081967213114754" src="/upload/94c9a7c1593f53757f2f05cbc4abc45f.png" data-type="png" data-w="976" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> 三种计数方式 </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">执行效果</strong>：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">count(*)包括了所有的列，相当于行数，在统计结果的时候，不会忽略列值为NULL</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">count(1)包括了忽略所有列，用1代表代码行，在统计结果的时候，不会忽略列值为NULL</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">count(列名)只包括列名那一列，在统计结果的时候，会忽略列值为空（这里的空不是只空字符串或者0，而是表示null）的计数，即某个字段值为NULL时，不统计。</p> </section></li> </ul> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">执行速度</strong>：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 列名为主键，count(列名)会比count(1)快 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 列名不为主键，count(1)会比count(列名)快 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果表多个列并且没有主键，则 count（1） 的执行效率优于 count（*） </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果有主键，则 select count（主键）的执行效率是最优的 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果表只有一个字段，则 select count（*）最优。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">13.一条SQL查询语句的执行顺序？</span></h2> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img data-ratio="0.16828929068150209" src="/upload/7c8fb99f71d766e63167257d5689bf44.png" data-type="png" data-w="1438" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> 查询语句执行顺序 </figcaption> </figure> <ol data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: decimal;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">FROM</strong>：对FROM子句中的左表<left_table>和右表<right_table>执行笛卡儿积（Cartesianproduct），产生虚拟表VT1</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">ON</strong>：对虚拟表VT1应用ON筛选，只有那些符合<join_condition>的行才被插入虚拟表VT2中</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">JOIN</strong>：如果指定了OUTER JOIN（如LEFT OUTER JOIN、RIGHT OUTER JOIN），那么保留表中未匹配的行作为外部行添加到虚拟表VT2中，产生虚拟表VT3。如果FROM子句包含两个以上表，则对上一个连接生成的结果表VT3和下一个表重复执行步骤1）～步骤3），直到处理完所有的表为止</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">WHERE</strong>：对虚拟表VT3应用WHERE过滤条件，只有符合<where_condition>的记录才被插入虚拟表VT4中</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">GROUP BY</strong>：根据GROUP BY子句中的列，对VT4中的记录进行分组操作，产生VT5</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">CUBE|ROLLUP</strong>：对表VT5进行CUBE或ROLLUP操作，产生表VT6</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">HAVING</strong>：对虚拟表VT6应用HAVING过滤器，只有符合<having_condition>的记录才被插入虚拟表VT7中。</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">SELECT</strong>：第二次执行SELECT操作，选择指定的列，插入到虚拟表VT8中</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">DISTINCT</strong>：去除重复数据，产生虚拟表VT9</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">ORDER BY</strong>：将虚拟表VT9中的记录按照<order_by_list>进行排序操作，产生虚拟表VT10。11）</p> </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;"><strong style="font-weight: bold;color: hsl(216, 80%, 44%);">LIMIT</strong>：取出指定行的记录，产生虚拟表VT11，并返回给查询用户</p> </section></li> </ol> <h1 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;color: black;font-size: 1.7em;font-weight: normal;border-bottom: 2px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="background: hsl(216, 100%, 68%);color: white;padding: 3px 10px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">数据库架构</span></h1> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">14.说说 MySQL 的基础架构?</span></h2> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.9368421052631579" src="/upload/59d9d5cf96b4aa7cb68a96122f8df48c.png" data-type="png" data-w="570" style="display: block;max-width: 100%;width: 90%;margin: 0 auto;box-shadow: #CCC 0 10px 15px;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> 在这里插入图片描述 </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;line-height: 26px;color: #666;">MySQL逻辑架构图主要分三层：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 客户端：最上层的服务并不是MySQL所独有的，大多数基于网络的客户端/服务器的工具或者服务都有类似的架构。比如连接处理、授权认证、安全等等。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> Server层：大多数MySQL的核心服务功能都在这一层，包括查询解析、分析、优化、缓存以及所有的内置函数（例如，日期、时间、数学和加密函数），所有跨存储引擎的功能都在这一层实现：存储过程、触发器、视图等。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 存储引擎层：第三层包含了存储引擎。存储引擎负责MySQL中数据的存储和提取。Server层通过API与存储引擎进行通信。这些接口屏蔽了不同存储引擎之间的差异，使得这些差异对上层的查询过程透明。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: normal;color: #333;font-size: 1.4em;border-bottom: 1px solid hsl(216, 100%, 68%);"><span style="display: none;"></span><span style="border-bottom: 1px solid hsl(216, 100%, 68%);">15.一条 SQL 查询语句在 MySQL 中如何执行的？</span></h2> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;color: black;list-style-type: disc;padding-left: 2em;" class="list-paddingleft-1"> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 先检查该语句 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">是否有权限</code>，如果没有权限，直接返回错误信息，如果有权限会先查询缓存 (MySQL8.0 版本以前)。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 如果没有缓存，分析器进行 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">语法分析</code>，提取 sql 语句中 select 等关键元素，然后判断 sql 语句是否有语法错误，比如关键词是否正确等等。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 语法解析之后，MySQL的服务器会对查询的语句进行优化，确定执行的方案。 </section></li> <li style="color: #666;"> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 完成查询优化后，按照生成的执行计划 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;color: hsl(216, 100%, 68%);">调用数据库引擎接口</code>，返回执行结果。 </section></li> </ul> <h1 data-tool="mdnice编辑器" style="margin-to
作者：微信小助手
<section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px;color: black;padding-right: 10px;padding-left: 10px;line-height: 1.6;letter-spacing: 0px;word-break: break-word;text-align: left;font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, "PingFang SC", Cambria, Cochin, Georgia, Times, "Times New Roman", serif;" data-mpa-powered-by="yiban.io"> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">前言</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">大家好，我是<strong>苏三</strong>。今天跟大家探讨一下分布式锁的设计与实现。希望对大家有帮助，如果有不正确的地方，欢迎指出，一起学习，一起进步哈~</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 分布式锁概述 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 数据库分布式锁 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> Redis分布式锁 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> Zookeeper分布式锁 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 三种分布式锁对比 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">1. 分布式锁概述</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">我们的系统都是分布式部署的，日常开发中，<strong>秒杀下单、抢购商品</strong>等等业务场景，为了防⽌库存超卖，都需要用到<strong>分布式锁</strong>。</p> <blockquote data-tool="mdnice编辑器" style="border-top: none;border-right: none;border-bottom: none;font-size: 0.9em;overflow: auto;color: rgb(106, 115, 125);padding: 10px 10px 10px 20px;margin-bottom: 20px;margin-top: 20px;border-left-color: rgb(239, 112, 96);background: rgb(255, 249, 249);"> <p style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;color: black;line-height: 26px;">分布式锁其实就是，控制分布式系统不同进程共同访问共享资源的一种锁的实现。如果不同的系统或同一个系统的不同主机之间共享了某个临界资源，往往需要互斥来防止彼此干扰，以保证一致性。</p> </blockquote> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">业界流行的分布式锁实现，一般有这3种方式：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 基于数据库实现的分布式锁 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 基于Redis实现的分布式锁 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;color: rgb(1, 1, 1);"> 基于Zookeeper实现的分布式锁 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;border-bottom: 2px solid rgb(239, 112, 96);font-size: 1.3em;"><span style="display: none;"></span><span style="display: inline-block;background: rgb(239, 112, 96);color: rgb(255, 255, 255);padding: 3px 10px 1px;border-top-right-radius: 3px;border-top-left-radius: 3px;margin-right: 3px;">2. 基于数据库的分布式锁</span><span style="display: inline-block;vertical-align: bottom;border-bottom: 36px solid #efebe9;border-right: 20px solid transparent;"> </span></h2> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 20px;"><span style="display: none;"></span>2.1 数据库悲观锁实现的分布式锁<span style="display: none;"></span></h3> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">可以使用<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">select ... for update </code>来实现分布式锁。我们自己的项目，<strong>分布式定时任务</strong>，就使用类似的实现方案，我给大家来展示个<strong>简单版的哈</strong></p> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">表结构如下：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/eytJa9K5jkoibl85XeZqwTZQyqEuWyMBibHSV0fuXWrjroYIqa2IOHHDFM03MI8Ky2Fib7q3MbRpSazXdqXmwibniakFib361Gm8Cs/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(39, 40, 34);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #272822;border-radius: 5px;">CREATE TABLE `t_resource_lock` (<br> `key_resource` varchar(45) COLLATE utf8_bin NOT NULL DEFAULT <span style="color: #a6e22e;line-height: 26px;">'资源主键'</span>,<br> `status` char(1) COLLATE utf8_bin NOT NULL DEFAULT <span style="color: #a6e22e;line-height: 26px;">''</span> COMMENT <span style="color: #a6e22e;line-height: 26px;">'S,F,P'</span>,<br> `lock_flag` int(10) unsigned NOT NULL DEFAULT <span style="color: #a6e22e;line-height: 26px;">'0'</span> COMMENT <span style="color: #a6e22e;line-height: 26px;">'1是已经锁 0是未锁'</span>,<br> `begin_time` datetime DEFAULT NULL COMMENT <span style="color: #a6e22e;line-height: 26px;">'开始时间'</span>,<br> `end_time` datetime DEFAULT NULL COMMENT <span style="color: #a6e22e;line-height: 26px;">'结束时间'</span>,<br> `client_ip` varchar(45) COLLATE utf8_bin NOT NULL DEFAULT <span style="color: #a6e22e;line-height: 26px;">'抢到锁的IP'</span>,<br> `time` int(10) unsigned NOT NULL DEFAULT <span style="color: #a6e22e;line-height: 26px;">'60'</span> COMMENT <span style="color: #a6e22e;line-height: 26px;">'方法生命周期内只允许一个结点获取一次锁，单位：分钟'</span>,<br> PRIMARY KEY (`key_resource`) USING BTREE<br>) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin<br></code></pre> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">加锁<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">lock</code>方法的伪代码如下：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block;background: url("https://mmbiz.qpic.cn/mmbiz_svg/eytJa9K5jkoibl85XeZqwTZQyqEuWyMBibHSV0fuXWrjroYIqa2IOHHDFM03MI8Ky2Fib7q3MbRpSazXdqXmwibniakFib361Gm8Cs/640?wx_fmt=svg") 10px 10px / 40px no-repeat rgb(39, 40, 34);height: 30px;width: 100%;margin-bottom: -7px;border-radius: 5px;"></span><code style="overflow-x: auto;padding: 16px;color: #ddd;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #272822;border-radius: 5px;">@Transcational //一定要加事务<br>public boolean lock(String <span style="color: rgb(221, 221, 221);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;font-size: 12px;text-align: left;background-color: rgb(39, 40, 34);white-space: pre-wrap;">keyRe</span><span style="color: rgb(221, 221, 221);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;font-size: 12px;text-align: left;background-color: rgb(39, 40, 34);white-space: pre-wrap;">source</span>，int time){<br> resourceLock = <span style="color: #a6e22e;line-height: 26px;">'select * from t_resource_lock where key_resource ='</span><span style="color: #75715e;line-height: 26px;">#{keySource}' for update';</span><br> <br> try{<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span>(resourceLock==null){<br> //插入锁的数据<br> resourceLock = new ResourceLock();<br> resourceLock.setTime(time);<br> resourceLock.setLockFlag(1); //上锁<br> resourceLock.setStatus(P); //处理中<br> resourceLock.setBeginTime(new Date());<br> int count = <span style="color: #a6e22e;line-height: 26px;">"insert into resourceLock"</span>; <br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span>(count==1){<br> //获取锁成功<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">true</span>;<br> }<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">false</span>;<br> }<br> }catch(Exception x){<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">false</span>;<br> }<br> <br> //没上锁并且锁已经超时，即可以获取锁成功<br> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span>(resourceLock.getLockFlag==<span style="color: #a6e22e;line-height: 26px;">'0'</span>&&<span style="color: #a6e22e;line-height: 26px;">'S'</span>.equals(resourceLock.getstatus)<br> && new Date()>=resourceLock.addDateTime(resourceLock.getBeginTime(,time)){<br> resourceLock.setLockFlag(1); //上锁<br> resourceLock.setStatus(P); //处理中<br> resourceLock.setBeginTime(new Date());<br> //update resourceLock;<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">true</span>;<br> }<span style="color: #f92672;font-weight: bold;line-height: 26px;">else</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">if</span>(new Date()>=resourceLock.addDateTime(resourceLock.getBeginTime(,time)){<br> //超时未正常执行结束,获取锁失败<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">false</span>;<br> }<span style="color: #f92672;font-weight: bold;line-height: 26px;">else</span>{<br> <span style="color: #a6e22e;line-height: 26px;">return</span> <span style="color: #f92672;font-weight: bold;line-height: 26px;">false</span>;<br> } <br>}<br><br></code></pre> <p data-tool="mdnice编辑器" style="padding-top: 8px;padding-bottom: 8px;line-height: 26px;">解锁<code style="font-size: 14px;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;color: rgb(239, 112, 96);">unlock</code>
作者：微信小助手
<section data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px;color: black;padding: 0 10px;line-height: 1.6;word-spacing: 0px;letter-spacing: 0px;word-break: break-word;word-wrap: break-word;text-align: left;font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, 'PingFang SC', Cambria, Cochin, Georgia, Times, 'Times New Roman', serif;" data-mpa-powered-by="yiban.io"> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.5504854368932038" src="/upload/371dc38368a2997488f06735da92b577.jpg" data-type="jpeg" data-w="1030" style="display: block;margin: 0 auto;max-width: 100%;"> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">公司项目是微服务架构，在服务发布的时候，上游时不时一堆报错，然后同事就一阵圈我。一开始百思不得其解，后来咨询了资深的同事才知道：<strong style="font-weight: bold;color: black;">原来是服务下线的时候没有优雅停机，没有去 注册将自己下线再停机，导致调用方拿到了旧的调用地址，导致调用失败！</strong> 看来优雅停机还是一个非常重要的知识点，可不能忽略，今天就让我们来盘盘它吧！</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 22px;letter-spacing: 5px;padding: 20px;color: rgb(0, 13, 131);text-align: center;background-image: url("https://mmbiz.qpic.cn/mmbiz_png/AVWicyZuuClEXUO6gIOiaRHJibxLTBRglfTvaXoIrYQr4nY6j3lJZ3SS9hHZzx5xLe5qxhJdd1yMIv1CQ8XLZFofQ/640?wx_fmt=png");background-repeat: no-repeat;background-attachment: initial;background-origin: initial;background-clip: initial;background-position: center -10px;background-size: 150px;"><span style="padding-bottom: 12px;border-bottom: 2px solid #000d83;">什么是优雅停机？</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">在 Linux 世界里，一切都是资源。当我们启动一个 JVM 的时候，我们就加载了许多的资源。而当我们关闭 JVM 的时候，JVM 只会释放内存这个资源，而其他资源是不会释放的，例如：网络连接、文件句柄等等。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">Linux 的网络连接数、文件句柄数都是有限的，如果我们没有及时释放，时间久了就会导致一些奇怪的问题。<strong style="font-weight: bold;color: black;">那么如何在 JVM 关闭的时候，释放这些资源呢？答案就是：利用 Java 提供的 ShutdownHook 接口。</strong> 我们所说的优雅停机，就是利用 Java 提供的 ShutdownHook 接口注册一个钩子，让 JVM 在关闭之前执行钩子函数的代码，让其关闭对应的资源。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 22px;letter-spacing: 5px;padding: 20px;color: rgb(0, 13, 131);text-align: center;background-image: url("https://mmbiz.qpic.cn/mmbiz_png/AVWicyZuuClEXUO6gIOiaRHJibxLTBRglfTvaXoIrYQr4nY6j3lJZ3SS9hHZzx5xLe5qxhJdd1yMIv1CQ8XLZFofQ/640?wx_fmt=png");background-repeat: no-repeat;background-attachment: initial;background-origin: initial;background-clip: initial;background-position: center -10px;background-size: 150px;"><span style="padding-bottom: 12px;border-bottom: 2px solid #000d83;">适用场景</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">在学会怎么使用优雅停机之前，我们需要弄清楚优雅停机适用于哪些场景，那我们就需要先弄清楚 JVM 关闭的几种情况了。JVM 关闭的情况可以分为 3 大类 11 个情况，如下图所示：</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7361809045226131" src="/upload/ee414c38a6ebd482355c1d9d7ab11e2b.png" data-type="png" data-w="1592" style="display: block;margin: 0 auto;max-width: 100%;"> <figcaption style="margin-top: 5px;text-align: center;color: #888;font-size: 14px;"> JVM 关闭的场景 </figcaption> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">在 JVM 关闭的 3 大类场景中，只有正常关闭与异常关闭是支持优雅停机的，而强制关闭则是不支持的。下面我们通过三个例子来验证一下。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">JVM 正常关闭</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">JVM 正常关闭这种情况，我们只需要正常运行一个 main 函数，然后为其注册一个 ShutdownHook 即可，其代码如下所示。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;"><span style="color: #c678dd;line-height: 26px;">public</span> <span style="line-height: 26px;"><span style="color: #c678dd;line-height: 26px;">class</span> <span style="color: #e6c07b;line-height: 26px;">NormalShutdownTest</span> </span>{<br> <span style="line-height: 26px;"><span style="color: #c678dd;line-height: 26px;">public</span> <span style="color: #c678dd;line-height: 26px;">void</span> <span style="color: #61aeee;line-height: 26px;">start</span><span style="line-height: 26px;">()</span> </span>{<br> Runtime.getRuntime().addShutdownHook(<span style="color: #c678dd;line-height: 26px;">new</span> Thread(() -><br> System.out.println(<span style="color: #98c379;line-height: 26px;">"钩子函数被执行，可以在这里关闭资源。"</span>)<br> ));<br> }<br><br> <span style="line-height: 26px;"><span style="color: #c678dd;line-height: 26px;">public</span> <span style="color: #c678dd;line-height: 26px;">static</span> <span style="color: #c678dd;line-height: 26px;">void</span> <span style="color: #61aeee;line-height: 26px;">main</span><span style="line-height: 26px;">(String[] args)</span> </span>{<br> <span style="color: #c678dd;line-height: 26px;">new</span> NormalShutdownTest().start();<br> System.out.println(<span style="color: #98c379;line-height: 26px;">"主应用程序在执行，正常关闭。"</span>);<br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">输出结果为：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">主应用程序在执行，正常关闭。<br>钩子函数被执行，可以在这里关闭资源。<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">可以看到钩子函数的代码正常执行了。如果你在 main 函数增加 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">System.exit(0)</code> 代码，执行之后的结果也还是一样。这说明 JVM 正常关闭情况下，是支持优雅停机的。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">异常关闭</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">JVM 异常关闭这种情况，我们尝试制造内存溢出。只需要声明一个 500 MB 的数组，然后设置 JVM 堆最大为 20 MB 即可（-Xmx20M），其代码如下所示。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">public class OomShutdownTest {<br> public void <span style="color: rgb(97, 174, 238);line-height: 26px;">start</span>() {<br> Runtime.getRuntime().addShutdownHook(new Thread(() -><br> System.out.println(<span style="color: #98c379;line-height: 26px;">"钩子函数被执行，可以在这里关闭资源"</span>)<br> ));<br> }<br><br> public static void main(String[] args) throws Exception {<br> new OomShutdownTest().start();<br> System.out.println(<span style="color: #98c379;line-height: 26px;">"主应用程序在执行，内存溢出关闭。"</span>);<br> byte[] b = new byte[500 * 1024 * 1024];<br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">执行结果为：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">主应用程序在执行，内存溢出关闭。<br>Exception <span style="color: #c678dd;line-height: 26px;">in</span> thread <span style="color: #98c379;line-height: 26px;">"main"</span> java.lang.OutOfMemoryError: Java heap space<br> at tech.shuyi.javacodechip.shutdownhook.OomShutdownTest.main(OomShutdownTest.java:13)<br>钩子函数被执行，可以在这里关闭资源<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">可以看到 JVM 抛出了 OOM 错误，但是钩子函数还是被执行了。如果你在 main 函数中自行抛出 RuntimeException，钩子函数也还是会被执行。感兴趣的朋友可以自行尝试一下。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">强制关闭</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">JVM 强制关闭这种情况，我们可以使用 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">Runtime.getRuntime().halt(1)</code> 进行测试，其代码如下所示。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">public class ForceShutdownTest {<br> public void <span style="color: rgb(97, 174, 238);line-height: 26px;">start</span>() {<br> Runtime.getRuntime().addShutdownHook(new Thread(() -><br> System.out.println(<span style="color: #98c379;line-height: 26px;">"钩子函数被执行，可以在这里关闭资源。"</span>)<br> ));<br> }<br><br> public static void main(String[] args) throws Exception {<br> new ForceShutdownTest().start();<br> System.out.println(<span style="color: #98c379;line-height: 26px;">"主应用程序在执行，强制关闭。"</span>);<br> Runtime.getRuntime().halt(1);<br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">执行结果：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">主应用程序在执行，强制关闭。<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">可以看到钩子函数并没有被执行，所以 JVM 强制关闭这种场景不支持优雅停机。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 22px;letter-spacing: 5px;padding: 20px;color: rgb(0, 13, 131);text-align: center;background-image: url("https://mmbiz.qpic.cn/mmbiz_png/AVWicyZuuClEXUO6gIOiaRHJibxLTBRglfTvaXoIrYQr4nY6j3lJZ3SS9hHZzx5xLe5qxhJdd1yMIv1CQ8XLZFofQ/640?wx_fmt=png");background-repeat: no-repeat;background-attachment: initial;background-origin: initial;background-clip: initial;background-position: center -10px;background-size: 150px;"><span style="padding-bottom: 12px;border-bottom: 2px solid #000d83;">最佳实践</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">看了上面的例子，看起来优雅停机没那么复杂嘛。实际上，优雅停机用不好，很可能出现一些其他问题。这里给出几个最佳实践原则，帮助大家用好优雅停机！</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">只注册一个钩子</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">我们都知道 JVM 可以注册多个钩子，而钩子本质上是一个线程，可以并发执行。那么就很可能出现钩子之间相互依赖，这样就会导致依赖死锁了。另外，也可能因为多个钩子操作同一个资源，导致资源竞争出现死锁。<strong style="font-weight: bold;color: black;">因此，较好的一种方式就是只注册一个钩子，所有的资源释放都在这个钩子中操作。</strong></p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">确保线程安全</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">因为钩子本质上也是一个线程，JVM 可能会并发执行多个钩子，JVM 并不保证它们的执行顺序，因此需要保证钩子中的操作是线程安全的。当然了，如果你只有一个钩子的话，那这个提示可以忽略了。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">不要做耗时的操作</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">在钩子中，不要做耗时的操作。因为当我们要关闭 JVM 时，用户肯定是希望尽快关闭，因此钩子中主要用于关闭残留资源，不应该再做其他耗时的操作。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">不要做注册、移除钩子的操作</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">在关闭钩子中，不能执行注册、移除钩子的操作，否则 JVM 抛出 IllegalStateException。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">不要调用 System.exit () 操作</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">也不能调用 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">System.exit()</code> 操作，但是调用 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">Runtime.halt()</code> 操作是可以的。我想，这是因为调用 System.exit () 操作会导致循环进入钩子，导致死循环吧。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">需要考虑的资源</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">除了上面一些代码上的操作需要考虑，我们还需要注意下面这些场景的处理：</p> <ul data-tool="mdnice编辑器" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;color: black;list-style-type: disc;" class="list-paddingleft-1"> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 池化资源的释放：数据库连接池、HTTP 连接池、线程池。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 在处理线程的释放：已经被连接的 HTTP 请求。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> MQ 消费者的处理：正在处理的消息。 </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;line-height: 26px;text-align: left;color: rgb(1,1,1);font-weight: 500;"> 隐形受影响的资源的处理：Zookeeper、Nacos 实例下线等。 </section></li> </ul> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 22px;letter-spacing: 5px;padding: 20px;color: rgb(0, 13, 131);text-align: center;background-image: url("https://mmbiz.qpic.cn/mmbiz_png/AVWicyZuuClEXUO6gIOiaRHJibxLTBRglfTvaXoIrYQr4nY6j3lJZ3SS9hHZzx5xLe5qxhJdd1yMIv1CQ8XLZFofQ/640?wx_fmt=png");background-repeat: no-repeat;background-attachment: initial;background-origin: initial;background-clip: initial;background-position: center -10px;background-size: 150px;"><span style="padding-bottom: 12px;border-bottom: 2px solid #000d83;">应用案例</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">Java 提供的优雅停机机制，可以说是许多框架的基础。诸如 Spring、Consul 等中间件框架，都是利用 Java 提供的这个机制进行优雅停机的。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">Spring 的优雅停机</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">例如 Spring 是基于 Java 语言开发的框架，那其也势必依赖于 JVM 的 ShutdownHook。Spring 关于优雅停机的代码在 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">org.springframework.context.support.AbstractApplicationContext#registerShutdownHook</code> 处，代码如下图所示。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">@Override<br>public void <span style="color: rgb(97, 174, 238);line-height: 26px;">registerShutdownHook</span>() {<br> <span style="color: #c678dd;line-height: 26px;">if</span> (this.shutdownHook == null) {<br> // No shutdown hook registered yet.<br> this.shutdownHook = new Thread(SHUTDOWN_HOOK_THREAD_NAME) {<br> @Override<br> public void <span style="color: rgb(97, 174, 238);line-height: 26px;">run</span>() {<br> synchronized (startupShutdownMonitor) {<br> doClose();<br> }<br> }<br> };<br> // 增加 ShutdownHook 钩子<br> Runtime.getRuntime().addShutdownHook(this.shutdownHook);<br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">可以看到 Spring 在 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">registerShutdownHook()</code> 函数里，注册了一个关闭的钩子，钩子中调用了 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">doClose()</code> 方法。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;padding: 0px;font-weight: bold;color: black;font-size: 20px;">服务治理的优雅停机</h3> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">不论是 Dubbo 还是 Spring Cloud 的分布式服务框架，需要关注的是怎么能在服务停止前，先将提供者在注册中心进行反注册，然后在停止服务提供者，这样才能保证业务系统不会产生各种 503、timeout 等现象。为了实现上述说到的效果，那么我们就必须关注优雅停机这件事情。</p> <h2 data-tool="mdnice编辑器" style="margin-top: 30px;margin-bottom: 15px;font-weight: bold;font-size: 22px;letter-spacing: 5px;padding: 20px;color: rgb(0, 13, 131);text-align: center;background-image: url("https://mmbiz.qpic.cn/mmbiz_png/AVWicyZuuClEXUO6gIOiaRHJibxLTBRglfTvaXoIrYQr4nY6j3lJZ3SS9hHZzx5xLe5qxhJdd1yMIv1CQ8XLZFofQ/640?wx_fmt=png");background-repeat: no-repeat;background-attachment: initial;background-origin: initial;background-clip: initial;background-position: center -10px;background-size: 150px;"><span style="padding-bottom: 12px;border-bottom: 2px solid #000d83;">彩蛋</span></h2> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">我们都知道通过 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">kill -15</code> 可以让 JVM 优雅停机，那我们是否可以监听特定的信号量，从而让程序做特定的操作呢？例如：让 JVM 监听第 12 信号量，然后打印一条日志，随后优雅停机。</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">答案是当然可以啦！我们只需要利用 Signal 类，并实现一个 SignHandler 类就可以了。其实现代码如下所示：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">public class CustomShutdownTest {<br> public void <span style="color: rgb(97, 174, 238);line-height: 26px;">start</span>() {<br> Runtime.getRuntime().addShutdownHook(new Thread(() -><br> System.out.println(<span style="color: #98c379;line-height: 26px;">"钩子函数被执行，可以在这里关闭资源。"</span>)<br> ));<br> }<br><br> public static void main(String[] args) {<br> // custom signal <span style="color: #e6c07b;line-height: 26px;">kill</span><br> Signal sg = new Signal(<span style="color: #98c379;line-height: 26px;">"USR2"</span>); // <span style="color: #e6c07b;line-height: 26px;">kill</span> -12 pid<br> Signal.handle(sg, new <span style="color: rgb(97, 174, 238);line-height: 26px;">SignalHandler</span>() {<br> @Override<br> public void handle(Signal signal) {<br> System.out.println(<span style="color: #98c379;line-height: 26px;">"接收到信号量："</span> + signal.getName());<br> // 监听信号量，通过System.exit(0)正常关闭JVM，触发关闭钩子执行收尾工作<br> System.exit(0);<br> }<br> });<br> // other logic<br> new CustomShutdownTest().start();<br> System.out.println(<span style="color: #98c379;line-height: 26px;">"主应用程序在执行，正常关闭。"</span>);<br> try {<br> Thread.sleep(30000);<br> } catch (InterruptedException e) {<br> e.printStackTrace();<br> }<br> }<br>}<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">我们启动该类后，先让其休眠 30 秒，随后用 jps 命令找到进程 ID，随后运行 <code style="font-size: 14px;word-wrap: break-word;padding: 2px 4px;border-radius: 4px;margin: 0 2px;color: #1e6bb8;background-color: rgba(27,31,35,.05);font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;word-break: break-all;">kill -USR2 PID</code> 即可，如截图所示。</p> <figure data-tool="mdnice编辑器" style="margin: 0;margin-top: 10px;margin-bottom: 10px;display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7213930348258707" src="/upload/74ca87559eb7e5d9271fef040b290fb0.jpg" data-type="jpeg" data-w="201" style="display: block;margin: 0 auto;max-width: 100%;"> </figure> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">随后可以看到控制台打印出如下消息：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;border-radius: 5px;box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><code style="overflow-x: auto;padding: 16px;color: #abb2bf;display: -webkit-box;font-family: Operator Mono, Consolas, Monaco, Menlo, monospace;font-size: 12px;-webkit-overflow-scrolling: touch;padding-top: 15px;background: #282c34;border-radius: 5px;">主应用程序在执行，正常关闭。<br>接收到信号量：USR2<br>钩子函数被执行，可以在这里关闭资源。<br></code></pre> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;">从上面消息我们知道，JVM 成功接收到了 USR2 信号量，也成功执行了钩子函数。搞定！</p> <p data-tool="mdnice编辑器" style="font-size: 16px;padding-top: 8px;padding-bottom: 8px;margin: 0;color: black;line-height: 2em;"><em style="font-style: italic;color: black;">提示：其实 USR2 是 Linux 第 12 个信号量，是留给用户使用的一个信号量。我们可以通过该信号量做一些定制化操作，从而实现更加复杂的功能。</em></p> </section>
作者：微信小助手
<section style="font-size: 15px;line-height: 1.6;box-sizing: border-box;font-style: normal;font-weight: 400;text-align: justify;"> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgba(0, 0, 0, 0.5);font-size: 14px;text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">vivo 互联网服务器团队- Shuai Guangying</p> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 1px;border-style: solid;border-color: rgb(160, 160, 160);padding: 10px;box-sizing: border-box;"> <section style="text-align: left;box-sizing: border-box;" powered-by="xiumi.us"> <section style="font-size: 14px;text-align: justify;line-height: 1.8;padding: 0px 5px;color: rgb(160, 160, 160);box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">本文梳理了Elasticsearch对于数值索引实现方案的升级和优化思考，从2015年至今数值索引的方案经历了多个版本的迭代，实现思路从最初的字符串模拟到KD-Tree，技术越来越复杂，能力越来越强大，应用场景也越来越丰富。从地理位置信息建模到多维坐标，数据检索到数据分析洞察都可以看到Elasticsearch的身影。</p> </section> </section> <section style="margin: 0px 0% -5px;text-align: right;line-height: 1;font-size: 5px;transform: translate3d(5px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="width: 0px;display: inline-block;vertical-align: top;border-bottom: 0.6em solid rgb(160, 160, 160);border-right: 0.6em solid rgb(160, 160, 160);box-sizing: border-box;border-top: 0.6em solid transparent !important;border-left: 0.6em solid transparent !important;"> <svg viewbox="0 0 1 1" style="float:left;line-height:0;width:0;vertical-align:top;"></svg> </section> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">一、业务背景</p> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">LBS服务是当前互联网重要的一环，涉及餐饮、娱乐、打车、零售等场景。在这些场景中，有很重要的一项基础能力：搜索附近的POI。比如搜索附近的美食，搜索附近的电影院，搜索附近的专车，搜索附近的门店。例如：以某个坐标点为中心查询出1km半径范围的POI坐标，如下图所示：</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.9525316455696202" data-s="300,640" src="/upload/93fe9af4444265722c224432aaa3babc.png" data-type="png" data-w="632" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">Elasticsearch在地理位置信息检索上具备了毫秒级响应的能力，而毫秒级响应对于用户体验至关重要。上面的问题使用Elasticsearch，只需用到geo_distance查询就可以解决业务问题。使用Elasticsearch的查询语法如下：</p> <section class="code-snippet__fix code-snippet__js"> <ul class="code-snippet__line-index code-snippet__js"> </ul> <pre class="code-snippet__js" data-lang="javascript"><code><span class="code-snippet_outer">GET /my_locations/_search</span></code><code><span class="code-snippet_outer">{</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"query"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"bool"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"must"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"match_all"</span>: {}</span></code><code><span class="code-snippet_outer"> },</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"filter"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"geo_distance"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"distance"</span>: <span class="code-snippet__string">"1km"</span>,</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"pin.location"</span>: {</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"lat"</span>: <span class="code-snippet__number">40</span>,</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__string">"lon"</span>: <span class="code-snippet__number">116</span></span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer">}</span></code></pre> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">工具的使用是一个非常简单的事情，更有意思在于工具解决问题背后的思想。理解了处理问题的思想，就可以超然于工具本身，做到举一反三。本文基于在海量数据背景下，如何实现毫秒级搜索附近的POI这个问题，探讨了Elasticsearch的实现方案，以及实现地理位置索引技术的演进过程。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">二、背景知识</p> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">在介绍Elasticsearch的处理方案前，我们首先需要介绍一些背景知识，主要是3个问题。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><strong style="box-sizing: border-box;">1. 如何精确定位一个地址？ </strong></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">由经度、纬度和相对高度组成的地理坐标系，能够明确标示出地球上的任何一个位置。地球上的经度范围[-180， 180]，纬度范围[-90，90]。通常以本初子午线(经度为0)、赤道(纬度为0)为分界线。对于大多数业务场景，由经纬度组成的二维坐标已经足以应对业务问题，可能重庆山城会有些例外。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><strong style="box-sizing: border-box;">2. 如何计算两个地址距离？</strong></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">对于平面坐标系，由勾股定理可以方便计算出两个点的距离。但是由于地球是一个不完美球体，且不同位置有不同海拔高度，所以精确计算两个距离位置是一个非常复杂的问题。在不考虑高度的情况下，二维坐标距离通常使用Haversine公式。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">这个公式非常简单，只需用到arcsin和cos两个高中数学公式。其中φ和λ表示两个点纬度和经度的弧度制度量。其中d即为所求两个点的距离，对应的数学公式如下(参考维基百科)：</p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.2345803842264914" data-s="300,640" src="/upload/96292ce5fed9eea101a0d1aa3c072da0.png" data-type="png" data-w="989" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">程序员更喜欢看代码，对照代码理解公式更简单。相应的代码如下：</p> <section class="code-snippet__fix code-snippet__js"> <ul class="code-snippet__line-index code-snippet__js"> </ul> <pre class="code-snippet__js" data-lang="java"><code><span class="code-snippet_outer"><span class="code-snippet__comment">// 代码摘自lucene-core-8.2.0， SloppyMath工具类</span></span></code><code><span class="code-snippet_outer"> </span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">/**</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * Returns the Haversine distance in meters between two points</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * given the previous result from {<span class="code-snippet__doctag">@link</span> #haversinSortKey(double, double, double, double)}</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@return</span> distance in meters.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> */</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__function"><span class="code-snippet__keyword">public</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">double</span> <span class="code-snippet__title">haversinMeters</span><span class="code-snippet__params">(<span class="code-snippet__keyword">double</span> sortKey)</span> </span>{</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">return</span> TO_METERS * <span class="code-snippet__number">2</span> * asin(Math.min(<span class="code-snippet__number">1</span>, Math.sqrt(sortKey * <span class="code-snippet__number">0.5</span>)));</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> </span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">/**</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * Returns a sort key for distance. This is less expensive to compute than</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * {<span class="code-snippet__doctag">@link</span> #haversinMeters(double, double, double, double)}, but it always compares the same.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * This can be converted into an actual distance with {<span class="code-snippet__doctag">@link</span> #haversinMeters(double)}, which</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * effectively does the second half of the computation.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> */</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__function"><span class="code-snippet__keyword">public</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">double</span> <span class="code-snippet__title">haversinSortKey</span><span class="code-snippet__params">(<span class="code-snippet__keyword">double</span> lat1, <span class="code-snippet__keyword">double</span> lon1, <span class="code-snippet__keyword">double</span> lat2, <span class="code-snippet__keyword">double</span> lon2)</span> </span>{</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">double</span> x1 = lat1 * TO_RADIANS;</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">double</span> x2 = lat2 * TO_RADIANS;</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">double</span> h1 = <span class="code-snippet__number">1</span> - cos(x1 - x2);</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">double</span> h2 = <span class="code-snippet__number">1</span> - cos((lon1 - lon2) * TO_RADIANS);</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">double</span> h = h1 + cos(x1) * cos(x2) * h2;</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">// clobber crazy precision so subsequent rounding does not create ties.</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">return</span> Double.longBitsToDouble(Double.doubleToRawLongBits(h) & <span class="code-snippet__number">0xFFFFFFFFFFFFFFF8L</span>);</span></code><code><span class="code-snippet_outer"> }</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">// haversin</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">// <span class="code-snippet__doctag">TODO:</span> remove these for java 9, they fixed Math.toDegrees()/toRadians() to work just like this.</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">public</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">final</span> <span class="code-snippet__keyword">double</span> TO_RADIANS = Math.PI / <span class="code-snippet__number">180</span>D;</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">public</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">final</span> <span class="code-snippet__keyword">double</span> TO_DEGREES = <span class="code-snippet__number">180</span>D / Math.PI;</span></code><code><span class="code-snippet_outer"> </span></code><code><span class="code-snippet_outer"> <span class="code-snippet__comment">// Earth's mean radius, in meters and kilometers; see http://earth-info.nga.mil/GandG/publications/tr8350.2/wgs84fin.pdf</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">private</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">final</span> <span class="code-snippet__keyword">double</span> TO_METERS = <span class="code-snippet__number">6_371_008.7714</span>D; <span class="code-snippet__comment">// equatorial radius</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">private</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">final</span> <span class="code-snippet__keyword">double</span> TO_KILOMETERS = <span class="code-snippet__number">6_371.0087714</span>D; <span class="code-snippet__comment">// equatorial radius</span></span></code><code><span class="code-snippet_outer"> </span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment">/**</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * Returns the Haversine distance in meters between two points</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * specified in decimal degrees (latitude/longitude). This works correctly</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * even if the dateline is between the two points.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <p></span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * Error is at most 4E-1 (40cm) from the actual haversine distance, but is typically</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * much smaller for reasonable distances: around 1E-5 (0.01mm) for distances less than</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * 1000km.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> *</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@param</span> lat1 Latitude of the first point.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@param</span> lon1 Longitude of the first point.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@param</span> lat2 Latitude of the second point.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@param</span> lon2 Longitude of the second point.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> * <span class="code-snippet__doctag">@return</span> distance in meters.</span></span></code><code><span class="code-snippet_outer"><span class="code-snippet__comment"> */</span></span></code><code><span class="code-snippet_outer"> <span class="code-snippet__function"><span class="code-snippet__keyword">public</span> <span class="code-snippet__keyword">static</span> <span class="code-snippet__keyword">double</span> <span class="code-snippet__title">haversinMeters</span><span class="code-snippet__params">(<span class="code-snippet__keyword">double</span> lat1, <span class="code-snippet__keyword">double</span> lon1, <span class="code-snippet__keyword">double</span> lat2, <span class="code-snippet__keyword">double</span> lon2)</span> </span>{</span></code><code><span class="code-snippet_outer"> <span class="code-snippet__keyword">return</span> haversinMeters(haversinSortKey(lat1, lon1, lat2, lon2));</span></code><code><span class="code-snippet_outer"> }</span></code></pre> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><strong style="box-sizing: border-box;">3. 如何方便在互联网分享经纬度坐标？</strong></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Geohash是2008-02-26由Gustavo Niemeyer在自己的个人博客上公布的算法服务。其初衷在于通过对经纬度的编码对外提供简短的URL标识地图位置，方便在电子邮件、论坛和网站中使用。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">实际上Geohash的价值不仅仅是提供简短的URL，它更大的价值在于：</p> </section> <section style="margin-top: 10px;margin-bottom: 10px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 1px;border-style: solid;border-color: transparent;padding: 10px;background-color: rgb(239, 239, 239);box-sizing: border-box;"> <ol class="list-paddingleft-1" style="list-style-type: decimal;box-sizing: border-box;" powered-by="xiumi.us"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Geohash给地图上每个坐标提供了独一无二的ID，这个唯一ID就相当于给每个地理位置提供了一个身份证。唯一ID在数据库中应用场景非常丰富。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">在数据库中给坐标点提供了另一种存储方式，将二维的坐标点转化成为一维的字符串，对于一维数据就可以借助B树等索引来加速查询。</p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;">Geohash是一种前缀编码，位置相近的坐标点前缀相同。通过前缀提供了高性能的邻近位置POI查询，而邻近位置POI查询是LBS服务的核心能力。</p></li> </ol> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">关于Geohash的编码规则，这里不展开。这里最关键的点在于：</p> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgba(0, 0, 0, 0.5);text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Geohash是一种前缀编码，位置相近的坐标点前缀相同。Geohash编码长度不同，所覆盖区域范围不同。</p> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">在前面知识的铺垫下，最简单的求一个坐标点指定半径范围内的坐标集合的方案就出炉了。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;" powered-by="xiumi.us"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">暴力算法</strong></p></li> </ul> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 1px;border-style: solid;border-color: rgb(160, 160, 160);padding: 10px;box-sizing: border-box;"> <section style="text-align: left;box-sizing: border-box;" powered-by="xiumi.us"> <section style="font-size: 14px;text-align: justify;line-height: 1.8;padding: 0px 5px;color: rgb(160, 160, 160);box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><span style="color: rgb(62, 62, 62);font-size: 15px;box-sizing: border-box;">中心坐标点依次跟集合中每个坐标点计算距离，筛选出符合半径条件的坐标点。</span><br style="box-sizing: border-box;"></p> </section> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">这个算法大家太熟悉了，就是最常见的暴力<strong style="box-sizing: border-box;">(Brute Force</strong>)算法。这个算法在海量数据背景下是没法满足毫秒级响应时间要求的，所以多用于离线计算。对于毫秒级响应的业务诉求，这个算法可以基于geohash进行改造。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <ul class="list-paddingleft-1" style="list-style-type: disc;box-sizing: border-box;" powered-by="xiumi.us"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">二次筛选</strong></p></li> </ul> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 1px;border-style: solid;border-color: rgb(160, 160, 160);padding: 10px;box-sizing: border-box;"> <section style="text-align: left;box-sizing: border-box;" powered-by="xiumi.us"> <section style="text-align: justify;line-height: 1.8;padding: 0px 5px;color: rgb(160, 160, 160);box-sizing: border-box;"> <ol class="list-paddingleft-1" style="list-style-type: decimal;box-sizing: border-box;"> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><span style="color: rgb(62, 62, 62);box-sizing: border-box;">基于坐标中心点计算出geohash, 基于半径确定geohash前缀。</span><br style="box-sizing: border-box;"></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><span style="color: rgb(62, 62, 62);box-sizing: border-box;">通过Geohash前缀初筛出大致符合要求的坐标点(需要将中心点所在Geohash块周围8个Geohash块纳入初筛范围)。</span></p></li> <li style="box-sizing: border-box;"><p style="margin: 0px;padding: 0px;box-sizing: border-box;"><span style="color: rgb(62, 62, 62);box-sizing: border-box;">对于初筛结果使用Haversine公式进行二次筛选。</span></p></li> </ol> </section> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">除了上述方案，Elasticsearch在地理信息处理上有哪些奇思妙想呢？</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding: 3px;display: inline-block;border-bottom: 1px solid rgb(65, 94, 255);font-size: 17px;color: rgb(65, 94, 255);box-sizing: border-box;"> <p style="margin: 0px;padding: 0px;box-sizing: border-box;">三、方案演进</p> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">Elasticsearch从2.0版本开始支持geo_distance查询，到当前已更新到7.14版本。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">从2015年至今已经经历了6年的发展， 建设了如下的能力：<br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.6656934306569343" data-s="300,640" src="/upload/3890ad164be0490b7c3c39bb99db6ad4.jpg" data-type="jpeg" data-w="685" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">技术迭代大致可以分为3个阶段：</p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.5292397660818714" data-s="300,640" src="/upload/84b748ded5741e5ff181a45ea980ad1e.jpg" data-type="jpeg" data-w="684" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">发展的成效显著，从性能测试的结果可以略窥一二：</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.3" data-s="300,640" src="/upload/759e71e809a65591a45775749fb29190.png" data-type="png" data-w="1280" style=""></p> <p style="text-align: center;margin-bottom: 0em;"><br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.28984375" data-s="300,640" src="/upload/c6dd0cd1dcbe3c72a28c44dbef3f6295.png" data-type="png" data-w="1280" style=""></p> <p style="text-align: center;margin-bottom: 0em;"><br></p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.29140625" data-s="300,640" src="/upload/b4ccbfdc6fef5cf4d0f73f9ad5fc19f7.png" data-type="png" data-w="1280" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">总的来说，资源消耗降低的前提下搜索和写入数据效率都有大幅度提升。下面就详细介绍Elasticsearch对地理信息索引的思路。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><span style="font-size: 16px;color: rgb(65, 95, 255);box-sizing: border-box;">3.1 史前时代</span></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">Elasticsearch是基于Lucene构建的搜索引擎。Lucene最开始的设想是一个全文检索工具箱，即支持字符串检索，并没有考虑数值类型的处理。其核心思想非常简单，将文档分词后，为每个词构建一个term => array[docIds]的映射。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">这样用户输入关键词只需要三步就可以获得想要的结果：</p> </section> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgb(62, 62, 62);text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">第一步: </strong>通过关键词找到对应的倒排表。这一步简单来说就是查词典。例如：</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">TermQuery.TermWeight 获取该term的倒排表，读取docId+freq信息。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">第二步: </strong>根据倒排表得到的docId和词频信息对文档进行打分，返回给用户分值最高的TopN结果。例如：TopScoreDocCollector -- collect()方法，基于小顶堆，保留分数最大的TopN文档。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">第三步: </strong>基于docId查询正排表获取文档字段明细信息。</p> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">这三步看起来简单，但简直是数据结构应用最佳战场，它需要综合考虑磁盘、内存、IO、数据结构时间复杂度，非常具有挑战性。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgba(0, 0, 0, 0.5);text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">例如：查词典可以用很多数据结构实现，比如跳跃表，平衡树、HashMap等，而Lucene的核心工程师Mike McCandless实现了一个只有他自己能懂的FST, 是综合了有限自动机和前缀树的一种数据结构，用来平衡查询复杂度和存储空间，比HashMap慢，但是空间消耗低。文档打分通常用小顶堆来维护分值最高的N个结果，如果有新的文档打分超过堆顶，则替换堆顶元素即可。 </p> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><strong style="box-sizing: border-box;">问题：</strong>对于真实业务场景而言，只有字符串匹配查询是不够的，字符串和数值是应用最广泛的两种数据类型。如果需要进行区间查询怎么办呢？这是一个数据库产品非常基础的能力。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">Lucene提供了一种适配方案RangeQuery。就是用枚举来模拟数值查询。简单来说：</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">RangeQuery=BooleanQuery+TermQuery，所以限制查询是整数且区间最大不能超过1024。这种实现是可以说是非常鸡肋的，好在Lucene 2.9.0版本真正支持数值查询。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgb(62, 62, 62);text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">LUCENE-1470,LUCENE-1582,LUCENE-1602,LUCENE-1673,LUCENE-1701, LUCENE-1712</p> </section> </section> </section> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 1px;border-style: solid;border-color: rgb(160, 160, 160);padding: 10px;box-sizing: border-box;"> <section style="text-align: left;box-sizing: border-box;" powered-by="xiumi.us"> <section style="text-align: justify;line-height: 1.8;padding: 0px 5px;color: rgb(160, 160, 160);box-sizing: border-box;"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><span style="color: rgb(62, 62, 62);box-sizing: border-box;">Added NumericRangeQuery and NumericRangeFilter, a fast alternative to RangeQuery/RangeFilter for numeric searches. They depend on a specific structure of terms in the index that can be created by indexing using the new NumericField or NumericTokenStream classes. NumericField can only be used for indexing and optionally stores the values as string representation in the doc store. Documents returned from IndexReader/IndexSearcher will return only the String value using the standard Fieldable interface. NumericFields can be sorted on and loaded into the FieldCache. (Uwe Schindler, Yonik Seeley, Mike McCandless)</span><br style="box-sizing: border-box;"></p> </section> </section> </section> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">这个实现很强大，支持了int/long/float/double/</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">short/byte，也不限制查询区间了。它的核心思路是将数值字节数组化，然后利用前缀分层管理区间。</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"> 如下图所示：</p> <p style="text-align: center;margin-bottom: 0em;"><img class="rich_pages wxw-img js_insertlocalimg" data-ratio="0.6865037194473964" data-s="300,640" src="/upload/fa68e21b6346bd08fe1c020f60c93d21.png" data-type="png" data-w="941" style=""></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br></p> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">本质上还是RangeQuery=</p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us">BooleanQuery+TermQuery，只不过在前面做了一层转换：通过前缀树管理一个区间实现了匹配词数量的缩减，而这个缩减是非常有效的。所以这里就有一个专家参数：precisionStep。就是用来控制每个数值字段在分词是生成term的数量，生成term数量越多，区间控制粒度越细，占用磁盘空间越大，查询效率通常越高。 </p> <section style="margin: 10px 0% 8px;text-align: left;justify-content: flex-start;display: flex;flex-flow: row nowrap;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-left: 3px solid rgb(219, 219, 219);border-bottom-left-radius: 0px;padding: 0px 0px 0px 8px;align-self: flex-start;flex: 0 0 auto;box-sizing: border-box;"> <section style="color: rgba(0, 0, 0, 0.5);text-align: justify;box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">例如：如果precisionStep=8,则意味前缀树叶子节点的上层控制着255个叶子。那么，当查询范围在1~511时，由于跨了相邻的2个非叶子节点，所以需要遍历511个term。但是假如查询范围在0~512，又只需遍历2个term即可。这样的实现用起来真的有过山车的感觉。</p> </section> </section> </section> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;"><strong style="box-sizing: border-box;">综上</strong>，Elasticsearch核心的Lucene倒排索引是一种经典的以不变应万变：字符串和数值索引核心都是查倒排表。理解这个核心，对于后面理解地理位置数据存储和查询非常关键。接下来我们以geo_distance的实现思路为探索主线条，探索一下ES各个版本的实现思路。</p> </section> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><span style="font-size: 16px;color: rgb(65, 95, 255);box-sizing: border-box;">3.2 Elasticsearch 2.0 版本</span></p> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;" powered-by="xiumi.us"><br style="box-sizing: border-box;"></p> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <p style="white-space: normal;margin: 0px;padding: 0px;box-sizing: border-box;">这个版本实现geo_distance查询的思路非常朴素，是建立在数值区间查询(NumericRangeQuery)的基础上。它的geo_point类型字段其实是一个复合字段，或者�
作者：微信小助手
<h2 data-tool="mdnice编辑器"><span style="font-size: 20px;"><strong>推送服务</strong></span></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">还记得一年半前，做的一个项目需要用到 Android 推送服务。和 iOS 不同，Android 生态中没有统一的推送服务。Google 虽然有 Google Cloud Messaging ，但是连国外都没统一，更别说国内了，直接被墙。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">所以之前在 Android 上做推送大部分只能靠轮询。而我们之前在技术调研的时候，搜到了 jPush 的博客，上面介绍了一些他们的技术特点，他们主要做的其实就是移动网络下的长连接服务。单机 50W-100W 的连接的确是吓我一跳！后来我们也采用了他们的免费方案，因为是一个受众面很小的产品，所以他们的免费版够我们用了。一年多下来，运作稳定，非常不错！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">时隔两年，换了部门后，竟然接到了一项任务，优化公司自己的长连接服务端。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">再次搜索网上技术资料后才发现，相关的很多难点都被攻破，网上也有了很多的总结文章，单机 50W-100W 的连接完全不是梦，其实人人都可以做到。但是光有连接还不够，QPS 也要一起上去。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">所以，这篇文章就是汇总一下利用 Netty 实现长连接服务过程中的各种难点和可优化点。</p> <h2 data-tool="mdnice编辑器"><strong><span style="font-size: 20px;">Netty 是什么</span></strong></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;visibility: visible;">Netty: http://netty.io/</p> <blockquote data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;padding: 10px 10px 10px 20px;border-left-color: rgb(178, 174, 197);color: rgb(106, 115, 125);white-space: normal;outline: 0px;text-align: left;border-top: none;border-right: none;border-bottom: none;overflow: auto;background: rgb(249, 249, 249);visibility: visible;"> <p style="outline: 0px;font-size: 16px;color: rgb(102, 102, 102);visibility: visible;"><a target="_blank" href="http://mp.weixin.qq.com/s?__biz=MzAwMTE3MDY4MQ==&mid=2652456570&idx=1&sn=5bc46c6f9580617705ac30291d8a9e86&chksm=81302becb647a2fa8485adb00f77a84e33ea94731aacdd36fec570fd2fafa42c53fa3fb21388&scene=21#wechat_redirect" textvalue="Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients." linktype="text" imgurl="" imgdata="null" data-itemshowtype="0" tab="innerlink" style="font-size: 14px;color: rgb(0, 0, 0);visibility: visible;" data-linktype="2" hasload="1" wah-hotarea="click"><span style="font-size: 14px;color: rgb(0, 0, 0);visibility: visible;">Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients.</span></a><span style="font-size: 14px;visibility: visible;"></span></p> </blockquote> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">官方的解释最精准了，其中最吸引人的就是高性能了。但是很多人会有这样的疑问：直接用 NIO 实现的话，一定会更快吧？就像我直接手写 JDBC 虽然代码量大了点，但是一定比 iBatis 快！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">但是，如果了解 Netty 后你才会发现，这个还真不一定！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">利用 Netty 而不用 NIO 直接写的优势有这些：</p> <ul data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;width: 577.412px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 高性能高扩展的架构设计，大部分情况下你只需要关注业务而不需要关注架构 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Zero-Copy</code> 技术尽量减少内存拷贝 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 为 Linux 实现 Native 版 Socket </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 写同一份代码，兼容 java 1.7 的 NIO2 和 1.7 之前版本的 NIO </section></li> <li> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Pooled Buffers</code> 大大减轻 <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Buffer</code> 和释放 <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Buffer</code> 的压力 </section> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> <br> </section> <section> <br> </section></li> </ul> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">特性太多，大家可以去看一下《Netty in Action》这本书了解更多。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">另外，Netty 源码是一本很好的教科书！大家在使用的过程中可以多看看它的源码，非常棒！</p> <h2 data-tool="mdnice编辑器"><span style="font-size: 20px;"><strong>瓶颈是什么</strong></span></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">想要做一个长链服务的话，最终的目标是什么？而它的瓶颈又是什么？</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">其实目标主要就两个：</p> <ol data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;width: 577.412px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 更多的连接 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 更高的 QPS </section></li> </ol> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">所以，下面就针对这两个目标来说说他们的难点和注意点吧。</p> <h2 data-tool="mdnice编辑器"><strong><span style="font-size: 20px;">更多的连接</span></strong></h2> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>非阻塞 IO</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">其实无论是用 Java NIO 还是用 Netty，达到百万连接都没有任何难度。因为它们都是非阻塞的 IO，不需要为每个连接创建一个线程了。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">欲知详情，可以搜索一下<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">BIO</code>,<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">NIO</code>,<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">AIO</code>的相关知识点。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>Java NIO 实现百万连接</strong></span></h3> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"><code style="padding: 16px;outline: 0px;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;">ServerSocketChannel ssc = ServerSocketChannel.open(); <br style="outline: 0px;">Selector sel = Selector.open(); <br style="outline: 0px;"> <br style="outline: 0px;">ssc.configureBlocking(<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">false</span>); <br style="outline: 0px;">ssc.socket().bind(<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> InetSocketAddress(<span style="outline: 0px;line-height: 26px;">8080</span>)); <br style="outline: 0px;">SelectionKey key = ssc.register(sel, SelectionKey.OP_ACCEPT); <br style="outline: 0px;"> <br style="outline: 0px;"><span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">while</span>(<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">true</span>) { <br style="outline: 0px;"> sel.select(); <br style="outline: 0px;"> Iterator it = sel.selectedKeys().iterator(); <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">while</span>(it.hasNext()) { <br style="outline: 0px;"> SelectionKey skey = (SelectionKey)it.next(); <br style="outline: 0px;"> it.remove(); <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">if</span>(skey.isAcceptable()) { <br style="outline: 0px;"> ch = ssc.accept(); <br style="outline: 0px;"> } <br style="outline: 0px;"> } <br style="outline: 0px;">} <br style="outline: 0px;"></code></pre> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">这段代码只会接受连过来的连接，不做任何操作，仅仅用来测试待机连接数极限。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">大家可以看到这段代码是 NIO 的基本写法，没什么特别的。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>Netty 实现百万连接</strong></span></h3> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"><code style="padding: 16px;outline: 0px;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;">NioEventLoopGroup bossGroup = <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> NioEventLoopGroup(); <br style="outline: 0px;">NioEventLoopGroup workerGroup= <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> NioEventLoopGroup(); <br style="outline: 0px;">ServerBootstrap bootstrap = <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> ServerBootstrap(); <br style="outline: 0px;">bootstrap.group(bossGroup, workerGroup); <br style="outline: 0px;"> <br style="outline: 0px;">bootstrap.channel( NioServerSocketChannel<span style="outline: 0px;line-height: 26px;">.<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">class</span>)</span>; <br style="outline: 0px;"> <br style="outline: 0px;">bootstrap.childHandler(<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> ChannelInitializer<SocketChannel>() { <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">@Override</span> <span style="outline: 0px;line-height: 26px;"><span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">protected</span> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">void</span> <span style="outline: 0px;color: rgb(166, 226, 46);font-weight: bold;line-height: 26px;">initChannel</span><span style="outline: 0px;line-height: 26px;">(SocketChannel ch)</span> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">throws</span> Exception </span>{ <br style="outline: 0px;"> ChannelPipeline pipeline = ch.pipeline(); <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">//todo: add handler </span><br style="outline: 0px;"> }}); <br style="outline: 0px;">bootstrap.bind(<span style="outline: 0px;line-height: 26px;">8080</span>).sync(); <br style="outline: 0px;"></code></pre> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">这段其实也是非常简单的 Netty 初始化代码。同样，为了实现百万连接根本没有什么特殊的地方。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>瓶颈到底在哪</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">上面两种不同的实现都非常简单，没有任何难度，那有人肯定会问了：实现百万连接的瓶颈到底是什么？</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">其实只要 java 中用的是非阻塞 IO（NIO 和 AIO 都算），那么它们都可以用单线程来实现大量的 Socket 连接。不会像 BIO 那样为每个连接创建一个线程，因为代码层面不会成为瓶颈。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">其实真正的瓶颈是在 Linux 内核配置上，默认的配置会限制全局最大打开文件数(Max Open Files)还会限制进程数。所以需要对 Linux 内核配置进行一定的修改才可以。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">这个东西现在看似很简单，按照网上的配置改一下就行了，但是大家一定不知道第一个研究这个人有多难。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>如何验证</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">让服务器支持百万连接一点也不难，我们当时很快就搞定了一个测试服务端，但是最大的问题是，我怎么去验证这个服务器可以支撑百万连接呢？</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">我们用 Netty 写了一个测试客户端，它同样用了非阻塞 IO ，所以不用开大量的线程。但是一台机器上的端口数是有限制的，用<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">root</code>权限的话，最多也就 6W 多个连接了。所以我们这里用 Netty 写一个客户端，用尽单机所有的连接吧。</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"><code style="padding: 16px;outline: 0px;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;">NioEventLoopGroup workerGroup = <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> NioEventLoopGroup(); <br style="outline: 0px;">Bootstrap b = <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> Bootstrap(); <br style="outline: 0px;">b.group(workerGroup); <br style="outline: 0px;">b.channel( NioSocketChannel<span style="outline: 0px;line-height: 26px;">.<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">class</span>)</span>; <br style="outline: 0px;"> <br style="outline: 0px;">b.handler(<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">new</span> ChannelInitializer<SocketChannel>() { <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">@Override</span> <br style="outline: 0px;"> <span style="outline: 0px;line-height: 26px;"><span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">public</span> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">void</span> <span style="outline: 0px;color: rgb(166, 226, 46);font-weight: bold;line-height: 26px;">initChannel</span><span style="outline: 0px;line-height: 26px;">(SocketChannel ch)</span> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">throws</span> Exception </span>{ <br style="outline: 0px;"> ChannelPipeline pipeline = ch.pipeline(); <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">//todo:add handler </span><br style="outline: 0px;"> } <br style="outline: 0px;"> }); <br style="outline: 0px;"> <br style="outline: 0px;"><span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">for</span> (<span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">int</span> k = <span style="outline: 0px;line-height: 26px;">0</span>; k < <span style="outline: 0px;line-height: 26px;">60000</span>; k++) { <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">//请自行修改成服务端的IP </span><br style="outline: 0px;"> b.connect(<span style="outline: 0px;line-height: 26px;">127.0</span><span style="outline: 0px;line-height: 26px;">.0</span><span style="outline: 0px;line-height: 26px;">.1</span>, <span style="outline: 0px;line-height: 26px;">8080</span>); <br style="outline: 0px;">} <br style="outline: 0px;"></code></pre> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">代码同样很简单，只要连上就行了，不需要做任何其他的操作。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">这样只要找到一台电脑启动这个程序即可。这里需要注意一点，客户端最好和服务端一样，修改一下 Linux 内核参数配置。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>怎么去找那么多机器</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">按照上面的做法，单机最多可以有 6W 的连接，百万连接起码需要17台机器！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">如何才能突破这个限制呢？其实这个限制来自于网卡。我们后来通过使用虚拟机，并且把虚拟机的虚拟网卡配置成了桥接模式解决了问题。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">根据物理机内存大小，单个物理机起码可以跑4-5个虚拟机，所以最终百万连接只要4台物理机就够了。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>讨巧的做法</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">除了用虚拟机充分压榨机器资源外，还有一个非常讨巧的做法，这个做法也是我在验证过程中偶然发现的。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">根据 TCP/IP 协议，任何一方发送<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">FIN</code>后就会启动正常的断开流程。而如果遇到网络瞬断的情况，连接并不会自动断开。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">那我们是不是可以这样做？</p> <ol data-tool="mdnice编辑器" class="list-paddingleft-1" style="margin-top: 8px;margin-bottom: 8px;padding-left: 25px;width: 577.412px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 启动服务端，千万别设置 Socket 的 <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">keep-alive</code>属性，默认是不设置的 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 用虚拟机连接服务器 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 强制关闭虚拟机 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 修改虚拟机网卡的 MAC 地址，重新启动并连接服务器 </section></li> <li style="outline: 0px;"> <section style="margin-top: 5px;margin-bottom: 5px;outline: 0px;line-height: 26px;color: rgb(1, 1, 1);"> 服务端接受新的连接，并保持之前的连接不断 </section></li> </ol> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">我们要验证的是服务端的极限，所以只要一直让服务端认为有那么多连接就行了，不是吗？</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">经过我们的试验后，这种方法和用真实的机器连接服务端的表现是一样的，因为服务端只是认为对方网络不好罢了，不会将你断开。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">另外，禁用<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">keep-alive</code>是因为如果不禁用，Socket 连接会自动探测连接是否可用，如果不可用会强制断开。</p> <h2 data-tool="mdnice编辑器"><strong><span style="font-size: 20px;">更高的 QPS</span></strong></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">由于 NIO 和 Netty 都是非阻塞 IO，所以无论有多少连接，都只需要少量的线程即可。而且 QPS 不会因为连接数的增长而降低（在内存足够的前提下）。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">而且 Netty 本身设计得足够好了，Netty 不是高 QPS 的瓶颈。那高 QPS 的瓶颈是什么？</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">是数据结构的设计！</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>如何优化数据结构</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">首先要熟悉各种数据结构的特点是必需的，但是在复杂的项目中，不是用了一个集合就可以搞定的，有时候往往是各种集合的组合使用。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">既要做到高性能，还要做到一致性，还不能有死锁，这里难度真的不小…</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">我在这里总结的经验是，不要过早优化。优先考虑一致性，保证数据的准确，然后再去想办法优化性能。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">因为一致性比性能重要得多，而且很多性能问题在量小和量大的时候，瓶颈完全会在不同的地方。所以，我觉得最佳的做法是，编写过程中以一致性为主，性能为辅；代码完成后再去找那个 TOP1，然后去解决它！</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>解决 CPU 瓶颈</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">在做这个优化前，先在测试环境中去狠狠地压你的服务器，量小量大，天壤之别。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">有了压力测试后，就需要用工具来发现性能瓶颈了！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">我喜欢用的是 VisualVM，打开工具后看抽样器(Sample)，根据自用时间(Self Time (CPU))倒序，排名第一的就是你需要去优化的点了！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">备注：Sample 和 Profiler 有什么区别？前者是抽样，数据不是最准但是不影响性能；后者是统计准确，但是非常影响性能。如果你的程序非常耗 CPU，那么尽量用 Sample，否则开启 Profiler 后降低性能，反而会影响准确性。</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.72480181200453" src="/upload/a9525b21a2f406fe9ffa89b9c1dc7fbc.png" data-type="png" data-w="883" style="margin: 3px 0px;outline: 0px;border-radius: 0px 0px 5px 5px;display: block;object-fit: contain;box-shadow: rgb(132, 161, 168) 0px 10px 15px;box-sizing: border-box !important;visibility: visible !important;width: 677px !important;height: auto !important;inset: auto;"> <figcaption style="margin-top: 5px;outline: 0px;text-align: center;color: rgb(136, 136, 136);font-size: 12px;"> <br> </figcaption> </figure> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">还记得我们项目第一次发现的瓶颈竟然是<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">ConcurrentLinkedQueue</code>这个类中的<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">size()</code>方法。量小的时候没有影响，但是<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">Queue</code>很大的时候，它每次都是从头统计总数的，而这个<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">size()</code>方法我们又是非常频繁地调用的，所以对性能产生了影响。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;"><code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">size()</code>的实现如下：</p> <pre data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);"><code style="padding: 16px;outline: 0px;overflow-x: auto;background: rgb(39, 40, 34);color: rgb(221, 221, 221);display: -webkit-box;font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;border-radius: 0px;font-size: 12px;"><span style="outline: 0px;line-height: 26px;"><span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">public</span> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">int</span> <span style="outline: 0px;color: rgb(166, 226, 46);font-weight: bold;line-height: 26px;">size</span><span style="outline: 0px;line-height: 26px;">()</span> </span>{ <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">int</span> count = <span style="outline: 0px;line-height: 26px;">0</span>; <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">for</span> (Node<E> p = first(); p != <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">null</span>; p = succ(p)) <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">if</span> (p.item != <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">null</span>) <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(117, 113, 94);line-height: 26px;">// Collection.size() spec says to max out </span><br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">if</span> (++count == Integer.MAX_VALUE) <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">break</span>; <br style="outline: 0px;"> <span style="outline: 0px;color: rgb(249, 38, 114);font-weight: bold;line-height: 26px;">return</span> count; <br style="outline: 0px;">} <br style="outline: 0px;"></code></pre> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">后来我们通过额外使用一个<code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">AtomicInteger</code>来计数，解决了问题。但是分离后岂不是做不到高一致性呢？没关系，我们的这部分代码关心最终一致性，所以只要保证最终一致就可以了。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">总之，具体案例要具体分析，不同的业务要用不同的实现。</p> <h3 data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;font-weight: bold;font-size: 18px;white-space: normal;outline: 0px;text-align: left;background-color: rgb(255, 255, 255);color: rgb(35, 171, 127);"><span style="color: rgb(0, 0, 0);"><strong>解决 GC 瓶颈</strong></span></h3> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">GC 瓶颈也是 CPU 瓶颈的一部分，因为不合理的 GC 会大大影响 CPU 性能。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">这里还是在用 VisualVM，但是你需要装一个插件：VisualGC</p> <figure data-tool="mdnice编辑器" style="margin-top: 10px;margin-bottom: 10px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);font-size: 16px;text-align: left;background-color: rgb(255, 255, 255);display: flex;flex-direction: column;justify-content: center;align-items: center;"> <img class="rich_pages wxw-img" data-ratio="0.7456896551724138" src="/upload/f852e5a48d18c7d476c5e6a3aa3984f9.png" data-type="png" data-w="928" style="margin: 3px 0px;outline: 0px;border-radius: 0px 0px 5px 5px;display: block;object-fit: contain;box-shadow: rgb(132, 161, 168) 0px 10px 15px;box-sizing: border-box !important;visibility: visible !important;width: 677px !important;height: auto !important;inset: auto;"> <figcaption style="margin-top: 5px;outline: 0px;text-align: center;color: rgb(136, 136, 136);font-size: 12px;"> <br> </figcaption> </figure> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">有了这个插件后，你就可以直观的看到 GC 活动情况了。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">按照我们的理解，在压测的时候，有大量的 New GC 是很正常的，因为有大量的对象在创建和销毁。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">但是一开始有很多 Old GC 就有点说不过去了！</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">后来发现，在我们压测环境中，因为 Netty 的 QPS 和连接数关联不大，所以我们只连接了少量的连接。内存分配得也不是很多。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">而 JVM 中，默认的新生代和老生代的比例是1:2，所以大量的老生代被浪费了，新生代不够用。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">通过调整 <code style="margin-right: 2px;margin-left: 2px;padding: 2px 4px;outline: 0px;font-size: 14px;border-radius: 4px;color: rgb(30, 107, 184);background-color: rgba(27, 31, 35, 0.05);font-family: "Operator Mono", Consolas, Monaco, Menlo, monospace;word-break: break-all;">-XX:NewRatio</code> 后，Old GC 有了显著的降低。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">但是，生产环境又不一样了，生产环境不会有那么大的 QPS，但是连接会很多，连接相关的对象存活时间非常长，所以生产环境更应该分配更多的老生代。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">总之，GC 优化和 CPU 优化一样，也需要不断调整，不断优化，不是一蹴而就的。</p> <h2 data-tool="mdnice编辑器"><strong><span style="font-size: 20px;">其他优化</span></strong></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">如果你已经完成了自己的程序，那么一定要看看《Netty in Action》作者的这个网站：Netty Best Practices a.k.a Faster == Better。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">相信你会受益匪浅，经过里面提到的一些小小的优化后，我们的整体 QPS 提升了很多。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">最后一点就是，java 1.7 比 java 1.6 性能高很多！因为 Netty 的编写风格是事件机制的，看似是 AIO。可 java 1.6 是没有 AIO 的，java 1.7 是支持 AIO 的，所以如果用 java 1.7 的话，性能也会有显著提升。</p> <h2 data-tool="mdnice编辑器"><strong><span style="font-size: 20px;">最后成果</span></strong></h2> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">经过几周的不断压测和不断优化了，我们在一台16核、120G内存(JVM只分配8G)的机器上，用 java 1.6 达到了60万的连接和20万的QPS。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">其实这还不是极限，JVM 只分配了8G内存，内存配置再大一点连接数还可以上去；</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">QPS 看似很高，System Load Average 很低，也就是说明瓶颈不在 CPU 也不在内存，那么应该是在 IO 了！上面的 Linux 配置是为了达到百万连接而配置的，并没有针对我们自己的业务场景去做优化。</p> <p data-tool="mdnice编辑器" style="margin-top: 20px;margin-bottom: 20px;white-space: normal;outline: 0px;color: rgb(0, 0, 0);text-align: left;background-color: rgb(255, 255, 255);font-size: 16px;">因为目前性能完全够用，线上单机 QPS 最多才 1W，所以我们先把精力放在了其他地方。相信后面我们还会去继续优化这块的性能，期待 QPS 能有更大的突破！</p>