重寫Nacos服務發現：多個服務器如何跨命名空間，訪問公共服務？
一、問題背景
在開發某個公共應用時，筆者發現該公共應用的數據是所有測試環境（假設存在 dev/dev2/dev3）通用的。
這就意味著只需部署一個應用，就能滿足所有測試環境的需求；也意味著所有測試環境都需要調用該公共應用，而不同測試環境的應用注冊在不同的 Nacos 命名空間。
二、兩種解決方案
如果所有測試環境都需要調用該公共應用，有兩種可行的方案。第一種，將該公共服務同時注冊到不同的測試環境所對應的命名空間中。
第二種，將公共應用注冊到單獨的命名空間，不同的測試環境能夠跨命名空間訪問該應用。
三、詳細的問題解決過程
先行交代筆者的版本號配置。Nacos 客戶端版本號為 NACOS 1.4.1；Java 項目的 Nacos 版本號如下。
最初想法是將該公共應用同時注冊到多個命名空間下。在查找資料的過程中，團隊成員在 GitHub 上發現了一篇類似問題的博客分享：Registration Center: Can services in different namespaces be called from each other? #1176，原文傳送器：https://github.com/alibaba/nacos/issues/1176。
01 注冊多個命名空間
從該博客中，我們看到其他程序員朋友也遇到了類似的公共服務的需求。在本篇文章中，筆者將進一步分享實現思路以及示例代碼。
說明：以下代碼內容來自用戶 chuntaojun 的分享。
shareNamespace={namespaceId[:group]},{namespaceId[:group]}
@RunWith(SpringRunner.class)
@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SelectServiceInShareNamespace_ITCase {
private NamingService naming1;
private NamingService naming2;
@LocalServerPort
private int port;
@Before
public void init() throws Exception{
NamingBase.prepareServer(port);
if (naming1 == null) {
Properties properties = new Properties();
properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
naming1 = NamingFactory.createNamingService(properties);
Properties properties2 = new Properties();
properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
naming2 = NamingFactory.createNamingService(properties2);
}
while (true) {
if (!"UP".equals(naming1.getServerStatus())) {
Thread.sleep(1000L);
continue;
}
break;
}
}
@Test
public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException {
String service1 = randomDomainName();
String service2 = randomDomainName();
naming1.registerInstance(service1, "127.0.0.1", 90);
naming2.registerInstance(service2, "127.0.0.2", 90);
Thread.sleep(1000);
List<Instance> instances = naming1.getAllInstances(service2);
Assert.assertEquals(1, instances.size());
Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName()));
}
@Test
public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException {
String service1 = randomDomainName();
String service2 = randomDomainName();
naming2.registerInstance(service1, groupName, "127.0.0.1", 90);
naming3.registerInstance(service2, "127.0.0.2", 90);
Thread.sleep(1000);
List<Instance> instances = naming3.getAllInstances(service1);
Assert.assertEquals(1, instances.size());
Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName()));
Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName())));
}
}
進一步考慮后發現該解決方案可能不太契合當前遇到的問題。公司目前的開發測試環境有很多個，并且不確定以后會不會繼續增加。
如果每增加一個環境，都需要修改一次公共服務的配置，并且重啟一次公共服務，著實太麻煩了。倒不如反其道而行，讓其他的服務器實現跨命名空間訪問公共服務。
02 跨命名空間訪問
針對實際問題查找資料時，我們找到了類似的參考分享《重寫 Nacos 服務發現邏輯動態修改遠程服務IP地址》，原文傳送器：https://www.cnblogs.com/changxy-codest/p/14632574.html。
跟著博客思路看代碼，筆者了解到服務發現的主要相關類是 NacosNamingService, NacosDiscoveryProperties, NacosDiscoveryAutoConfiguration。
然后，筆者將博客的示例代碼復制過來，試著進行如下調試：
@Slf4j
@Configuration
@ConditionalOnNacosDiscoveryEnabled
@ConditionalOnProperty(
name = {"spring.profiles.active"},
havingValue = "dev"
)
@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})
public class DevEnvironmentNacosDiscoveryClient {
@Bean
@ConditionalOnMissingBean
public NacosDiscoveryProperties nacosProperties() {
return new DevEnvironmentNacosDiscoveryProperties();
}
static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties {
private NamingService namingService;
@Override
public NamingService namingServiceInstance() {
if (null != this.namingService) {
return this.namingService;
} else {
Properties properties = new Properties();
properties.put("serverAddr", super.getServerAddr());
properties.put("namespace", super.getNamespace());
properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName());
if (super.getEndpoint().contains(":")) {
int index = super.getEndpoint().indexOf(":");
properties.put("endpoint", super.getEndpoint().substring(0, index));
properties.put("endpointPort", super.getEndpoint().substring(index + 1));
} else {
properties.put("endpoint", super.getEndpoint());
}
properties.put("accessKey", super.getAccessKey());
properties.put("secretKey", super.getSecretKey());
properties.put("clusterName", super.getClusterName());
properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart());
try {
this.namingService = new DevEnvironmentNacosNamingService(properties);
} catch (Exception var3) {
log.error("create naming service error!properties={},e=,", this, var3);
return null;
}
return this.namingService;
}
}
}
static class DevEnvironmentNacosNamingService extends NacosNamingService {
public DevEnvironmentNacosNamingService(Properties properties) {
super(properties);
}
@Override
public List<Instance> selectInstances(String serviceName, List<String> clusters, boolean healthy) throws NacosException {
List<Instance> instances = super.selectInstances(serviceName, clusters, healthy);
instances.stream().forEach(instance -> instance.setIp("10.101.232.24"));
return instances;
}
}
}
調試后發現博客提供的代碼并不能滿足筆者的需求，還得進一步深入探索。
但幸運的是，調試過程發現 Nacos 服務發現的關鍵類是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery，其中的關鍵方法是 getInstances() 和 getServices()，即「返回指定服務 ID 的所有服務實例」和「獲取所有服務的名稱」。
也就是說，對 getInstances() 方法進行重寫肯定能實現本次目標——跨命名空間訪問公共服務。
/**
* Return all instances for the given service.
* @param serviceId id of service
* @return list of instances
* @throws NacosException nacosException
*/
public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
String group = discoveryProperties.getGroup();
List<Instance> instances = discoveryProperties.namingServiceInstance()
.selectInstances(serviceId, group, true);
return hostToServiceInstanceList(instances, serviceId);
}
/**
* Return the names of all services.
* @return list of service names
* @throws NacosException nacosException
*/
public List<String> getServices() throws NacosException {
String group = discoveryProperties.getGroup();
ListView<String> services = discoveryProperties.namingServiceInstance()
.getServicesOfServer(1, Integer.MAX_VALUE, group);
return services.getData();
}
03 最終解決思路及代碼示例
具體的解決方案思路大致如下：
- 生成一個共享配置類
NacosShareProperties，用來配置共享公共服務的namespace和group； - 重寫配置類
NacosDiscoveryProperties（新：NacosDiscoveryPropertiesV2），將新增的共享配置類作為屬性放進該配置類，后續會用到； - 重寫服務發現類
NacosServiceDiscovery（新：NacosServiceDiscoveryV2），這是最關鍵的邏輯； - 重寫自動配置類
NacosDiscoveryAutoConfiguration，將自定義相關類比 Nacos 原生類更早的注入容器。
最終代碼中用到了一些工具類，可以自行補充完整。
/**
* <pre>
* @description: 共享nacos屬性
* @author: rookie0peng
* @date: 2022/8/29 15:22
* </pre>
*/
@Configuration
@ConfigurationProperties(prefix = "nacos.share")
public class NacosShareProperties {
private final Map<String, Set<String>> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>();
/**
* 共享nacos實體列表
*/
private List<NacosShareEntity> entities;
public List<NacosShareEntity> getEntities() {
return entities;
}
public void setEntities(List<NacosShareEntity> entities) {
this.entities = entities;
}
public Map<String, Set<String>> getNamespaceGroupMap() {
safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
.forEach(entity -> {
Set<String> groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>());
if (nonNull(entity.getGroupNames()))
groupNames.addAll(entity.getGroupNames());
});
return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP);
}
@Override
public String toString() {
return "NacosShareProperties{" +
"entities=" + entities +
'}';
}
/**
* 共享nacos實體
*/
public static final class NacosShareEntity {
/**
* 命名空間
*/
private String namespace;
/**
* 分組
*/
private List<String> groupNames;
public String getNamespace() {
return namespace;
}
public void setNamespace(String namespace) {
this.namespace = namespace;
}
public List<String> getGroupNames() {
return groupNames;
}
public void setGroupNames(List<String> groupNames) {
this.groupNames = groupNames;
}
@Override
public String toString() {
return "NacosShareEntity{" +
"namespace='" + namespace + '\'' +
", groupNames=" + groupNames +
'}';
}
}
}
/**
* @description: naocs服務發現屬性重寫
* @author: rookie0peng
* @date: 2022/8/30 1:19
*/
public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties {
private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class);
private final NacosShareProperties nacosShareProperties;
private static final Map<String, NamingService> NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>();
public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) {
super();
this.nacosShareProperties = nacosShareProperties;
}
public Map<String, NamingService> shareNamingServiceInstances() {
if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) {
return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
}
List<NacosShareProperties.NacosShareEntity> entities = Optional.ofNullable(nacosShareProperties)
.map(NacosShareProperties::getEntities).orElse(Collections.emptyList());
entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
.filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace))
.forEach(entity -> {
try {
NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace()));
if (namingService != null) {
NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService);
}
} catch (Exception e) {
log.error("create naming service error! properties={}, e=", this, e);
}
});
return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
}
private Properties getNacosProperties(String namespace) {
Properties properties = new Properties();
properties.put(SERVER_ADDR, getServerAddr());
properties.put(USERNAME, Objects.toString(getUsername(), ""));
properties.put(PASSWORD, Objects.toString(getPassword(), ""));
properties.put(NAMESPACE, namespace);
properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName());
String endpoint = getEndpoint();
if (endpoint.contains(":")) {
int index = endpoint.indexOf(":");
properties.put(ENDPOINT, endpoint.substring(0, index));
properties.put(ENDPOINT_PORT, endpoint.substring(index + 1));
}
else {
properties.put(ENDPOINT, endpoint);
}
properties.put(ACCESS_KEY, getAccessKey());
properties.put(SECRET_KEY, getSecretKey());
properties.put(CLUSTER_NAME, getClusterName());
properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());
// enrichNacosDiscoveryProperties(properties);
return properties;
}
}
/**
* @description: naocs服務發現重寫
* @author: rookie0peng
* @date: 2022/8/30 1:10
*/
public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery {
private final NacosDiscoveryPropertiesV2 discoveryProperties;
private final NacosShareProperties nacosShareProperties;
private final NacosServiceManager nacosServiceManager;
public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) {
super(discoveryProperties, nacosServiceManager);
this.discoveryProperties = discoveryProperties;
this.nacosShareProperties = nacosShareProperties;
this.nacosServiceManager = nacosServiceManager;
}
/**
* Return all instances for the given service.
* @param serviceId id of service
* @return list of instances
* @throws NacosException nacosException
*/
public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
String group = discoveryProperties.getGroup();
List<Instance> instances = discoveryProperties.namingServiceInstance()
.selectInstances(serviceId, group, true);
if (isEmpty(instances)) {
Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
Map<String, NamingService> namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances();
for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
String namespace;
NamingService namingService;
if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
continue;
Set<String> groupNames = namespaceGroupMap.get(namespace);
List<Instance> shareInstances;
if (isEmpty(groupNames)) {
shareInstances = namingService.selectInstances(serviceId, group, true);
if (nonEmpty(shareInstances))
break;
} else {
shareInstances = new ArrayList<>();
for (String groupName : groupNames) {
List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
if (nonEmpty(subShareInstances)) {
shareInstances.addAll(subShareInstances);
}
}
}
if (nonEmpty(shareInstances)) {
instances = shareInstances;
break;
}
}
}
return hostToServiceInstanceList(instances, serviceId);
}
/**
* Return the names of all services.
* @return list of service names
* @throws NacosException nacosException
*/
public List<String> getServices() throws NacosException {
String group = discoveryProperties.getGroup();
ListView<String> services = discoveryProperties.namingServiceInstance()
.getServicesOfServer(1, Integer.MAX_VALUE, group);
return services.getData();
}
public static List<ServiceInstance> hostToServiceInstanceList(
List<Instance> instances, String serviceId) {
List<ServiceInstance> result = new ArrayList<>(instances.size());
for (Instance instance : instances) {
ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId);
if (serviceInstance != null) {
result.add(serviceInstance);
}
}
return result;
}
public static ServiceInstance hostToServiceInstance(Instance instance,
String serviceId) {
if (instance == null || !instance.isEnabled() || !instance.isHealthy()) {
return null;
}
NacosServiceInstance nacosServiceInstance = new NacosServiceInstance();
nacosServiceInstance.setHost(instance.getIp());
nacosServiceInstance.setPort(instance.getPort());
nacosServiceInstance.setServiceId(serviceId);
Map<String, String> metadata = new HashMap<>();
metadata.put("nacos.instanceId", instance.getInstanceId());
metadata.put("nacos.weight", instance.getWeight() + "");
metadata.put("nacos.healthy", instance.isHealthy() + "");
metadata.put("nacos.cluster", instance.getClusterName() + "");
metadata.putAll(instance.getMetadata());
nacosServiceInstance.setMetadata(metadata);
if (metadata.containsKey("secure")) {
boolean secure = Boolean.parseBoolean(metadata.get("secure"));
nacosServiceInstance.setSecure(secure);
}
return nacosServiceInstance;
}
private NamingService namingService() {
return nacosServiceManager
.getNamingService(discoveryProperties.getNacosProperties());
}
}
/**
* @description: 重寫nacos服務發現的自動配置
* @author: rookie0peng
* @date: 2022/8/30 1:08
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
@ConditionalOnNacosDiscoveryEnabled
@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})
public class NacosDiscoveryAutoConfigurationV2 {
@Bean
@ConditionalOnMissingBean
public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) {
return new NacosDiscoveryPropertiesV2(nacosShareProperties);
}
@Bean
@ConditionalOnMissingBean
public NacosServiceDiscovery nacosServiceDiscovery(
NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager
) {
return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager);
}
}
本以為問題到這就結束了，但最后自測時發現程序根本不走 Nacos 的服務發現邏輯，而是執行 Ribbon 的負載均衡邏輯com.netflix.loadbalancer.AbstractLoadBalancerRule。
不過實現類是 com.alibaba.cloud.nacos.ribbon.NacosRule，繼續基于 NacosRule 重寫負載均衡。
/**
* @description: 共享nacos命名空間規則
* @author: rookie0peng
* @date: 2022/8/31 2:04
*/
public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule {
private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class);
@Autowired
private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2;
@Autowired
private NacosShareProperties nacosShareProperties;
/**
* 重寫choose方法
*
* @param key
* @return
*/
@SneakyThrows
@Override
public Server choose(Object key) {
try {
String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName();
DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer();
String name = loadBalancer.getName();
NamingService namingService = nacosDiscoveryPropertiesV2
.namingServiceInstance();
List<Instance> instances = namingService.selectInstances(name, true);
if (CollectionUtils.isEmpty(instances)) {
LOGGER.warn("no instance in service {}, then to get share service's instance", name);
List<Instance> shareNamingService = this.getShareNamingService(name);
if (nonEmpty(shareNamingService))
instances = shareNamingService;
else
return null;
}
List<Instance> instancesToChoose = instances;
if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) {
List<Instance> sameClusterInstances = instances.stream()
.filter(instance -> Objects.equals(clusterName,
instance.getClusterName()))
.collect(Collectors.toList());
if (!CollectionUtils.isEmpty(sameClusterInstances)) {
instancesToChoose = sameClusterInstances;
}
else {
LOGGER.warn(
"A cross-cluster call occurs，name = {}, clusterName = {}, instance = {}",
name, clusterName, instances);
}
}
Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose);
return new NacosServer(instance);
}
catch (Exception e) {
LOGGER.warn("NacosRule error", e);
return null;
}
}
@Override
public void initWithNiwsConfig(IClientConfig iClientConfig) {
}
private List<Instance> getShareNamingService(String serviceId) throws NacosException {
List<Instance> instances = Collections.emptyList();
Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
Map<String, NamingService> namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances();
for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
String namespace;
NamingService namingService;
if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
continue;
Set<String> groupNames = namespaceGroupMap.get(namespace);
List<Instance> shareInstances;
if (isEmpty(groupNames)) {
shareInstances = namingService.selectInstances(serviceId, true);
if (nonEmpty(shareInstances))
break;
} else {
shareInstances = new ArrayList<>();
for (String groupName : groupNames) {
List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
if (nonEmpty(subShareInstances)) {
shareInstances.addAll(subShareInstances);
}
}
}
if (nonEmpty(shareInstances)) {
instances = shareInstances;
break;
}
}
return instances;
}
}
至此問題得以解決。
在 Nacos 上配置好共享 namespace 和 group 后，就能夠進行跨命名空間訪問了。
# nacos共享命名空間配置 示例
nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7b
nacos.share.entities[0].groupNames[0]=DEFAULT_GROUP
# 指定服務使用共享的負載均衡規則，service-id是注冊到nacos上的服務id，ShareNacosNamespaceRule需要寫全限定名
service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule
注意：如果 Java 項目的 nacos discovery 版本用的是 2021.1，則不需要重寫 Ribbon 的負載均衡類，因為該版本的 Nacos 不依賴 Ribbon。
2.2.1.RELEASE 版本的 nacos discovery 依賴 Ribbon.
2021.1 版本的 nacos discovery 不依賴 Ribbon。
四、總結
為了達到共享命名空間的預期，構思、查找資料、實現邏輯、調試，前后一共花費 4 天左右。
但該功能仍然存在共享服務緩存等可優化空間，留待后續實現。
五、參考文獻
[1] Registration Center: Can services in different namespaces be called from each other? [EB/OL]. https://github.com/alibaba/nacos/issues/1176, 2019-05-07/2022-11-29.
[2] 重寫Nacos服務發現邏輯動態修改遠程服務IP地址 [EB/OL]. https://www.cnblogs.com/changxy-codest/p/14632574.html, 2021-04-08/2022-11-29.
了解更多敏捷開發、項目管理、行業動態等消息，關注我們 [LigaAI@oschina](https://my.oschina.net/u/5057806) 或點擊LigaAI - 新一代智能研發協作平臺，在線申請體驗我們的產品。
695856371
| 喜歡本站的朋友可以收藏本站,或者加入我們大家一起來交流技術!
