FCIP简介(网关 原始信号)

Content

Introduction

Prerequisite(Requirement Components Used Convention)

Background Information

Configure(Network Diagram Configuration)

Verify

Troubleshoot

FCIP简介

FCIP：基于TCP/IP的光纤信道

（FCIP：Entire Fibre Channel Frame Over IP）

基于IP的光纤通道（FCIP）是连接TCP/IP网络链路上的光纤通道架构的一项IETF建议标准。该协议可以作为通过密集波分多路复用和暗光纤来连接存储区域网的一项替代选择。开发使用能够承担得起并且可以随时使用的IP服务，可以大大降低每个月的广域网连接成本并扩大光纤通道站点之间的最大距离。

FCIP通过在IP网络的两个端点之间建立一条隧道来传输光纤通道数据。在发送端，帧被封装到TCP/IP之中。在接收端，IP包解包之后得到的光纤通道帧被发送给目标结构。这种技术通常被称为隧道，它过去一直与AppleTalk和SNA等非IP协议一起使用。 该项技术是使用FCIP网关来实现的，它一般通过光纤通道交换机的扩展端口连接到每个SAN上。所有前往远程地点的存储业务均通过共同的隧道。接收端的光纤通道交换机负责引导每个帧前往适当的光纤通道端点设备。 多存储会话可以同时通过FCIP隧道，尽管在隧道的会话之间没有任何区别。从IP网络角度而言，FCIP隧道是不透明的。 IP网络管理工具可以在隧道的任何一侧监测网关，但不能监测在隧道内移动的个别光纤通道事务。因此这些工具在隧道的每侧都会观察到两个FCIP网关，但它们之间的通信就像是在单一源点和目的地之间，而不是在多个存储主机和目标之间。 连接光纤通道交换机可以创建一个类似于桥接LAN或其他二层网络的单一光纤通道架构。这意味着连接两个远程站点与FCIP网关可以创造一个延伸数英里的光纤通道架构。这样保存了远程站点之间的光纤通道操作，但可能使桥接fabric容易受到fabric重新配置或过度的基于fabric的广播的攻击。

网关

FCIP网关针对每条隧道链路通常成对出售。例如，将A站点与B站点连接起来将需要一对网关，而将A站点与C站点连接起来将需要额外的一对网关。FCIP更适合点对点连接，而非多点连接。 因为FCIP只是封装和解开IP中的光纤通道帧，所以厂商区别其网关的方法很少。因此某些制造商正在将FCIP网关减少为嵌入光纤通道交换机的刀片。 另一个建议的IETF标准是Internet光纤通道协议（iFCP），它使用了与FCIP同样的光纤通道帧封装方案。不过，iFCP是一项更复杂的协议，它旨在克服扩展架构的潜在弱点，实现多点部署，并向个别光纤通道事务提供本地IP寻址。 在管理上，FCIP使用了服务定位协议（SLP）来确定IP网络中的FCIP网关。由于FCIP网关较少，SLP提供了一种合适的检查表方法。iSCSI和iFCP可以使用SLP，但对于更复杂的环境，Internet存储名称服务器（iSNS）是首选。FCIP网关不支持iSNS。 在安全上，IPSec提供了验证、加密和数据完整性。FCIP也使用IPSec的自动密钥管理协议和Internet密钥管理来处理安全密钥的创建和管理。 FCIP标准预计将在一年之内得到最终批准。

FCIP 实体基本功能是实现利用 FC 帧封装技术转发 FC 帧。从 IP 网络角度看，FCIP 实体是对等结构并通过 TCP/IP 通信，在基于 IP 的网络中每个 FCIP 实体包含一个或多个 TCP 端点；从 FC 角度看，数对 FCIP 实体与它们相关联的 FC 实体相结合，在 FC 结构单元间转发 FC 帧，而 FC 终端并不知道 FCIP 链路的存在。

原始信号

FC 原始信号、原始序列以及第一类 FC 帧不经过 FCIP 链路传输，这是因为 FC 帧封装技术并不能为其编码。FC 封装帧的路径选择取决于 IP 网络正常的路由选择过程。

一个 FCIP 实体可能包含多个 FCIP 链路端点，而每个 FCIP 链路端点（FCIP_LEP）只能与另外的一个 FCIP_LEP 建立通信。FCIP 并不参与 FC 帧的路由选择过程。此外 FCIP 控制 & 服务模块（FCIP Control & Services module）可能用到 TCP/IP 服务质量（QOS）特性。

要发起通信，需要为每个 FCIP 实体静态或动态配置 IP 地址及响应 FCIP 实体的 TCP 端口号。如果当前 FCIP 实体支持动态发现，则使用服务定位协议（SLPv2）执行上述功能。在和对等 FCIP 实体创建一个 TCP 连接之前，FCIP 实体必须先要确认 IP 地址、TCP 端口、FC 结构实体的名字、CP 连接参数以及服务质量（QOS）等信息。

FCIP 实体并不参与 FC 源和目标标识符的发现过程， FC 地址的发现功能由 FC 体系中的有关协议和技术提供。

为保证 IP 网络的安全性， FCIP 实体必须： 1. 确保密码保护认证及数据加密的完整性； 2. 实施数据的机密性和安全性特性。

在单个 TCP 连接中，此规范通过 TCP/IP 按照发送顺序传送字节流。

<haha_sun > 加点原滋原味进来：

MDS to MDS 802.1Q Configuration with FCIP

协议结构 光纤信道帧封装头结构 ― FCIP 特定：

8162432bitProtocol #Version-Protocol #-Versionreplication of encapsulation word 0pFlagsReserved-pFlags-ReservedFlagsFrame Length-Flags-Frame LengthTime Stamp (integer)Time Stamp (fraction)CRC

通用字段：Protocol# ― 分配的 IANA 协议编号，用于识别采用 FCIP 封装的协议。 Version ― 封装版本，规定在 [ENCAP] 中。 Protocol# ― 补充协议#。 Version ― 补充版本。 Flags ― 封装标志。 Frame Length ― 包括整个 FC 封装帧的长度，其中包含 FC 封装头和32位字单元中的 FC 帧（包括 SOF 和 EOF 字）。 Flags ― 补充 Flags 字段。 Frame Length ― 补充 Frame Length 字段。 Time Stamp [integer] ― 帧 Time Stamp 的整数部分，规定在 [ENCAP] 中。 Time Stamp ― Time Stamp [fraction] 的小数部分，规定在 [ENCAP]。 CRC ― 头 CRC。必须对 iFCP 有效。

FCIP 特定字段：协议特定字段的第1个字包括 FC 帧封装中对第0个字的复制部分。 pFlags （协议特定标志）字段包含 FC 封装头中特定协议使用等信息。

ChReservedSF

SF （特定帧）位表明 FCIP 帧是封装 FC 帧还是 FSF （FCIP 特定帧）。 Ch （变换）位表示回音 FSF 是否已经被改变。如果 FSF 位不为1，那么 Ch 位为0。

Content

Introduction

Prerequisites

Requirements

Components Used

Conventions

Background Information

Configure

Network Diagram

Configurations

Verify

Troubleshoot

Note on Native VLAN Mismatch

Related Information

Introduction

This document provides a sample configuration for Fibre Channel Over TCP/IP (FCIP) with 802.1Q Multilayer Director Switch (MDS) to MDS.

FCIP describes mechanisms that allow the interconnection of islands of Fibre Channel (FC) storage area networks (SANs) over IP-based networks to form a unified SAN in a single FC fabric. FCIP relies on IP-based network services to provide the connectivity between the SAN islands over local area networks, metropolitan area networks, or wide area networks.

Fibre Channel SANs Connected by FCIP

FCIP uses Transmission Control Protocol (TCP) on port 3225 as a network layer transport.

Prerequisite

Requirement

The IP backbone must be operational and delivering the required bandwidth to support the applications running across the FCIP links—this could be a Layer 2 (L2) or Layer 3 (L3) topology. If it is a L3 topology, the intermediate routers or multilayer switches must be set up and configured to appropriately forward IP traffic between source and destination IP addresses of the FCIP tunnels. If Quality of Service (QoS)or traffic shaping is enforced at any network device in the path between the FCIP peers, the network manager administrating the IP infrastructure should be consulted to get the necessary details before configuring any TCP-related parameters and features on the Multilayer Director Switch (MDS) FCIP profile(s). The Ethernet switches which are adjacent to the MDSes must support and be configured for 802.1Q trunking if subinterfaces are configured on the MDS IP Storage (IPS) services module.

Components Used

The information in this document is based on these software and hardware versions:

MDS 9509 with IPS service module (DS-X9308-SMIP) running version 1.2.(2a)

MDS 9216 with IPS service module (DS-X9308-SMIP) running version 1.2.(2a)

Catalyst 6509 running Catalyst OS (CatOS) 7.4(3)

Win2003 Server (HPQ Pro-Liant-P4) with Emulex LP9K HBA

IBM Storage Array (ESS-2105-F20)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Convention

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Background Information

FCIP consists of these specifications:

ANSI T11FC-SW-2 describes the operation and interaction of FC switches including E_Port and fabric operation.

FC-BB-2 is a mapping that pertains to the extension of FC switched networks across a TCP network backbone, and defines reference models that support E_Port and B_Port.

IETF IPS Working GroupFC over TCP covers the TCP/IP requirements for transporting FC frames over an IP network.

FC frame encapsulation defines the common fibre encapsulation format.

IEEE 802 StandardsIEEE 802 LANs of all types may be connected together with MAC Bridges, as specified in ISO/IEC 15802-3. This standard defines the operation of VLAN Bridges that permit the definition, operation, and administration of VLAN topologies within a Bridged LAN infrastructure.

An interconnection between two SAN switches or fabrics across FCIP is called an FCIP link and can contain one or more TCP connections. Each end of a FCIP link is associated with a Virtual E port (VE_port) or a B_port, depending on the implementation. FC-BB and FC-BB-2 are describing the differences between both approaches. The IP services module (DS-X9308-SMIP) is supporting both modes but defaults to VE_Port, which is also the recommended mode to run if all relevant peers are DS-X9308-SMIP modules. The VE_Port functionality on MDS platforms is also supporting TE port functionality, which makes it capable of trunking traffic from multiple virtual SANs (VSANs) across one FCIP instance. The Gigabit Ethernet (GE) interfaces residing aboard the Cisco X9308-SMIP modules support 802.1Q in order to leverage the bandwidth of 1 Gbps between two or more FCIP tunnels in situations where low bandwidth requirements per FCIP tunnel exist. One must understand that sharing bandwidth by using dot1q is not providing deterministic Bandwidth per FCIP tunnel when FCIP Profile TCP parameters are left in default state.

Configure

On the MDSes, you need to familiarize yourself with the IPS configuration guides for both platforms. The most current version of the manuals can be found at Configuring IP Storage on Cisco.com. On the Ethernet switch side, one needs to be familiar with the dot1q trunking configuration specifics. In this particular example, a Catalyst running Hybrid CatOS is deployed; different configuration may apply to other Cisco switches or to switches from other vendors.

For Catalyst 6000 series running hybrid mode, refer to Configuring Ethernet VLAN Trunks. For Native IOS, refer to Configuring VLANs. For Catalyst XL type switches running Native IOS, refer to Configuring VLANs.

Note: To find additional information on the commands used in this document, use the Command Lookup Tool (registered customers only) .

Network Diagram

This document uses the network setup shown in the diagram below.

Topology 2

Topology 2 depicts one FCIP tunnel running across a 802.1Q trunk on either side of the IP cloud. The IP cloud is collapsed into one multilayer switch (Catalyst 6500) which routes traffic from VLAN 1000 to VLAN 500 and from VLAN 500 to VLAN 1000. VLAN 1000 conceptually maps to IP subnet 100.100.100.0/30, and VLAN 500 maps to IP subnet 100.100.100.4/30. The way the MDS maps and retrieves dot1q frames will become clear in the configuration section below. For simplicity, only one FCIP tunnel across one physical interface on both MDSes is defined; in reality, one would only use dot1q trunking to share the bandwidth of one Gigabit interface between multiple FCIP tunnels.

Configuration

MDS 9509 (Bison) with IPS-8 module

MDS 9216 (Canterbury) with IPS-8 module

Catalyst 6000 (Aspendos) with IPS-8 module

MDS 9509 (Bison) with IPS-8 modulebison# sh verCisco Storage Area Networking Operating System (SAN-OS) SoftwareTAC support: http://www.cisco.com/tacCopyright (c) 2002-2003 by Cisco Systems, Inc. All rights reserved.The copyright for certain works contained herein are owned byAndiamo Systems, Inc. and/or other third parties and are used anddistributed under license.SoftwareBIOS: version 1.0.8loader: version 1.2(2)kickstart: version 1.2(2a)system: version 1.2(2a)BIOS compile time: 08/07/03kickstart image file is: bootflash:/k122akickstart compile time: 9/23/2003 11:00:00system image file is: bootflash:/s122asystem compile time: 10/8/2003 18:00:00HardwareRAM 1024584 kBbootflash: 500736 blocks (block size 512b)slot0: 0 blocks (block size 512b)bison uptime is 1 days 15 hours 45 minute(s) 44 second(s)Last reset Reason: UnknownSystem version: 1.2(2a)Service: bison# sh runBuilding Configuration ...fcip profile 1ip address 100.100.100.1!--- FCIP profile 1 is bound to the local relevant IPS interface. !--- In this example, it is the IP address of interface Gig4/1.vsan databasevsan 200 name test vsan 600 vsan 601 fcdomain priority 1 vsan 1fcdomain domain 1 preferred vsan 1fcdomain domain 1 preferred vsan 600fcdomain domain 1 preferred vsan 601interface fcip1no shutdownswitchport trunk allowed vsan 600-601use-profile 1peer-info ipaddr 100.100.100.6<!--- peer IP address is the address on remote MDS Canterbury<!--- configured on interface Gig4/1.500.vsan databasevsan 600 interface fc3/1vsan 601 interface fc3/8boot system bootflash:/s122a sup-1boot kickstart bootflash:/k122a sup-1boot system bootflash:/s122a sup-2boot kickstart bootflash:/k122a sup-2boot asm-sfn bootflash:/ilc1.bin module 4ip domain-name cisco.comip name-server 144.254.10.123ip default-gateway 10.48.69.129ip route 100.100.100.4 255.255.255.252 100.100.100.2 distance 2!--- The next hop IP address is 100.100.100.2 !--- and is owned by the intermediate !--- Ethernet multilayer switch Aspendos.zone name z-fcip2 vsan 600member pwwn 50:05:07:63:00:d0:94:4cmember pwwn 10:00:00:00:c9:32:a6:e3zone name Zone_a1 vsan 601member pwwn 10:00:00:00:00:01:00:00member pwwn 10:00:00:00:00:05:00:00zone default-zone permit vsan 1zone default-zone permit vsan 603zoneset distribute full vsan 600zoneset name zs-fcip2 vsan 600member z-fcip2zoneset name Agilent_1 vsan 601member Zone_a1zoneset activate name zs-fcip2 vsan 600zoneset activate name Agilent_1 vsan 601interface fc3/1no shutdown!--- Output supressed.interface fc3/8no shutdowninterface mgmt0ip address 10.48.69.151 255.255.255.192interface GigabitEthernet4/1no shutdowninterface GigabitEthernet4/1.1000ip address 100.100.100.1 255.255.255.252switchport mtu 3000no shutdown!--- Here the subinterface 1000 is configured, !--- which ties into dot1q VLAN 1000 on the Ethernet switch.!--- The MTU size is changed from the default 1500 bytes to 3000, !--- because the intermediate switch supports jumbo frames !--- on both L2 and L3.MDS 9216 (Canterbury) with IPS-8 modulecanterbury# sh runBuilding Configuration ...fcip profile 1ip address 100.100.100.6vsan databasevsan 600 vsan 601 fcdomain domain 2 preferred vsan 600fcdomain domain 2 preferred vsan 601interface fcip1use-profile 1peer-info ipaddr 100.100.100.1vsan databasevsan 600 interface fc1/1vsan 601 interface fc1/8boot system bootflash:/s122a boot kickstart bootflash:/k122a fcalias name test vsan 1ip domain-name cisco.comip name-server 144.254.10.123ip default-gateway 10.48.69.129ip route 10.61.0.0 255.255.0.0ip route 10.61.0.0 255.255.0.0 10.48.69.200ip route 100.100.100.0 255.255.255.252 100.100.100.5 distance 2!--- Static IPS route required to reach the FCIP peer address.line vtyexec-timeout 0switchname canterburysystem default switchport trunk mode autozone name z-fcip2 vsan 600member pwwn 50:05:07:63:00:d0:94:4cmember pwwn 10:00:00:00:c9:32:a6:e3zone default-zone permit vsan 777zoneset distribute full vsan 600zoneset name zs-fcip2 vsan 600member z-fcip2zoneset activate name zs-fcip2 vsan 600zoneset activate name Agilent_1 vsan 601interface GigabitEthernet2/1no shutdowninterface GigabitEthernet2/1.500ip address 100.100.100.6 255.255.255.252switchport mtu 3000no shutdowninterface fc1/1no shutdowninterface fc1/8no shutdowninterface mgmt0ip address 10.48.69.156 255.255.255.128Catalyst 6000 (Aspendos) with IPS-8 moduleAspendos> (enable) sh vlan 500VLAN Name Status IfIndex Mod/Ports, Vlans---- -------------------------------- --------- ------- -------------------500 VLAN0500 active 191 1/1 3/3 15/1VLAN Type SAID MTU Parent RingNo BrdgNo Stp Brdg Mode Trans1 Trans2---- ---- ---- ---- ------ ------ ------ ------ ------ ------ ------ ------500 enet 100 500 1500 - - - - - 0 0VLAN MISTP-Inst DynCreated RSPAN---- ---------- ---------- --------500 - static disabledAspendos> (enable) sh vlan 1000VLAN Name Status IfIndex Mod/Ports, Vlans---- -------------------------------- --------- ------- -------------------1000 fcip-extra-hop active 131 3/7 15/1VLAN Type SAID MTU Parent RingNo BrdgNo Stp Brdg Mode Trans1 Trans2---- ---- ---- ---- ------ ------ ------ ------ ------ ------ ------ ------1000 enet 101 000 1500 - - - - - 0 0VLAN MISTP-Inst DynCreated RSPAN---- ---------- ---------- --------1000 - static disabledAspendos> (enable) sh trunk 3/3* - indicates vtp domain mismatchPort Mode Encapsulation Status Native vlan---- ---- ------------- ------------- -----------3/3 on dot1q trunking1Port Vlans allowed on trunk---- ----------------------------------------------------------------------3/3 1,500Port Vlans allowed and active in management domain ---- ----------------------------------------------------------------------3/3 1,500Port Vlans in spanning tree forwarding state and not pruned---- ----------------------------------------------------------------------3/3 1,500Aspendos> (enable) sh trunk 3/7* - indicates vtp domain mismatchPort Mode Encapsulation Status Native vlan---- ---- ------------- ------------ -----------3/7 on dot1q trunking1Port Vlans allowed on trunk---- ----------------------------------------------------------------------3/7 1,1000Port Vlans allowed and active in management domain ---- ----------------------------------------------------------------------3/7 1,1000Port Vlans in spanning tree forwarding state and not pruned---- ----------------------------------------------------------------------3/7 1,1000!--- Note that VLAN 500 is trunked on port 3/3, which is physically connected !--- to MDS Canterbury while port 3/7 trunks for VLAN 1000 and is connected !--- to MDS Bison. The port 15/1 is an internal port and directs to the !--- multilayer entitity in the switch.Aspendos> (enable) sh port jumbo Jumbo frames MTU size is 9216 bytes.Jumbo frames enabled on port(s) 1/2,3/1-16.!--- L2 Jumbo support must be enabled for all relevant ports.Aspendos> (enable) sh spantree 3/3Port Vlan Port-State Cost Prio Portfast Channel_id---- ---- ---------- ---- ---- -------- ----------3/3 1 forwarding 4 32 disabled 0 3/3 500 forwarding4 32 disabled 0 Aspendos> (enable) sh spantree 3/7Port Vlan Port-State Cost Prio Portfast Channel_id---- ---- ---------- ---- ---- -------- ----------3/7 1 forwarding 4 32 disabled 0 3/7 1000 forwarding4 32 disabled 0 Aspendos> (enable) sh spantree 15/1Port Vlan Port-State Cost Prio Portfast Channel_id---- ---- ---------- ---- ---- -------- ----------15/1 192 forwarding 4 32 enabled 0 15/1 500 forwarding4 32 enabled 0 15/1 691 forwarding 4 32 enabled 0 15/1 1000 forwarding4 32 enabled 0!--- All relevant ports on the Catalyst are forwarding !--- for their respective VLANs.Aspendos> (enable) ses 15Trying Router-15...Connected to Router-15.Escape character is '^]'.User Access VerificationPassword: Aspendos_MSFC2> enPassword: Aspendos_MSFC2#!interface Vlan500mtu 3000ip address 100.100.100.5 255.255.255.252endAspendos_MSFC2# sh run int vlan 1000Building configuration...Current configuration : 113 bytes!interface Vlan1000description "test-vlan-fcip-1000"mtu 3000ip address 100.100.100.2255.255.255.252end!--- The VLANs 500 and 1000 are configured properly on the MSFC !--- of the Catalyst 6000.!--- Both subnets are directly connected to the same Ethernet !--- switch, so no additional routes are needed here. Also note !--- that the MTU size is changed accordingly on the MDS Gigabit !--- interfaces to 3000 bytes. This change is required to support !--- jumbo multilayer switching.

Verify

This section provides information you can use to confirm that your configuration is working properly.

Certain show commands are supported by the Output Interpreter Tool (registered customers only) , which allows you to view an analysis of show command output.

show interface gig x/y —Displays status of the relevant Gigabit interface bound to the FCIP profile.

show ips stats tcp int gig x/y —Displays TCP statistics and active connections for the relevant Gigabit interface.

show ips arp int gig x/y —Displays all Address Resolution Protocol (ARP) entries for the relevant Gigabit interface; the next hop or peer should be present in this list.

show ips ip route int gig x/y —Displays the specific routes going across the relevant Gigabit interface.

show interface fcip x —Displays the FCIP interface status and all details related to this FCIP tunnel.

show profile fcip x —Displays the IP address to which the profile is bound and all configured TCP parameters.

show int fcip xcounters—Used to check if there are any frames going through the FCIP tunnel.

show fcdomain vsan x —Lists all domain-related details; used to verify that the fabric is formed across the FCIP tunnel(s).

show fcns da vsan x —Displays all pwwn, FC4-Types, and FCIDs of the relevant VSAN; used to verify that all expected entries are distributed across the FCIP tunnel(s).

Note: The dot1q interface acts as a normal physical interface for FCIP. There is no specific requirement to verify proper operation other than the show interface gig x/y.z command where z represents the subinterface. Depending upon the device connected to the local MDS Gigabit interfaces, you must consult the related documentation to verify that interfaces and ports are configured correctly.

Troubleshoot

Be sure to issue the show commands multiple times to build a counter history. Counters that are not related to a point in time and just collected only once are mostly useless.

Use the configurations shown below for more troubleshooting.

MDS 9509 (Bison)

MDS 9216 (Canterbury)

Note on Native VLAN Mismatch

MDS 9509 (Bison)bison# sh int gig 4/1.1000GigabitEthernet4/1.1000 is up Hardware is GigabitEthernet, address is 0005.3000.a85a Internet address is 100.100.100.1/30 MTU 3000 bytes!--- Configured to 3000 bytes.5 minutes input rate 488 bits/sec, 61 bytes/sec, 0 frames/sec 5 minutes output rate 488 bits/sec, 61 bytes/sec, 0 frames/sec 1785 packets input, 996030 bytes 0 multicast frames, 0 compressed 0 input errors, 0 frame, 0 overrun 0 fifo 1812 packets output, 354152 bytes, 0 underruns 0 output errors, 0 collisions, 0 fifo 0 carrier errors!--- MTU is configured to 3000 bytes to avoid unnecessary !--- TCP segmentation and limit overhead.bison# sh ips stats tcp int gig 4/1 deTCP Statistics for port GigabitEthernet4/1 TCP send stats 337202017 segments, 222637392068 bytes 130562402 data, 205533417 ack only packets 503 control (SYN/FIN/RST), 0 probes, 1105737 window updates 7 segments retransmitted, 2208 bytes 4 retransmitted while on ethernet send queue, 40061909 packets split 250922624 delayed acks sent TCP receive stats 932985742 segments, 921498012 data packets in sequence, 936715052100 bytes in sequence 770241 predicted ack, 856752348 predicted data 0 bad checksum, 0 multi/broadcast, 0 bad offset 0 no memory drops, 0 short segments 0 duplicate bytes, 16 duplicate packets 0 partial duplicate bytes, 0 partial duplicate packets 53128 out-of-order bytes, 165 out-of-order packets 0 packet after window, 0 bytes after window 5 packets after close 76225562 acks, 192030009160 ack bytes, 0 ack toomuch, 5851 duplicate acks 0 ack packets left of snd_una, 0 non-4 byte aligned packets 9124012 window updates, 0 window probe 1381 pcb hash miss, 984 no port, 103 bad SYN, 0 paws drops TCP Connection Stats 272 attempts, 107 accepts, 163 established 511 closed, 3 drops, 206 conn drops 3 drop in retransmit timeout, 20 drop in keepalive timeout 0 drop in persist drops, 0 connections drained TCP Miscellaneous Stats 61792500 segments timed, 76225541 rtt updated 124 retransmit timeout, 0 persist timeout 5760 keepalive timeout, 5740 keepalive probes TCP SACK Stats 0 recovery episodes, 0 data packets, 0 data bytes 0 data packets retransmitted, 0 data bytes retransmitted 0 connections closed, 0 retransmit timeouts TCP SYN Cache Stats 107 entries, 107 connections completed, 0 entries timed out 0 dropped due to overflow, 0 dropped due to RST 0 dropped due to ICMP unreach, 0 dropped due to bucket overflow 0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop 0 hash collisions, 0 retransmitted100.100.100.1:64860 100.100.100.6:3225 ESTABLISH 0 0100.100.100.1:64862 100.100.100.6:3225 ESTABLISH 0 0100.100.100.1:3225 0.0.0.0:0 LISTEN 0 0!--- No specific counters are maintained per subinterface.!--- All stats for all subinterfaces on Gig 4/1 are seen here.bison# sh ips arp interface gig 4/1Protocol Address Age (min) Hardware Addr Type InterfaceInternet 100.100.100.2 13 0008.e21e.c7bc ARPA GigabitEthernet4/1.1000 !--- No specific ARP table is maintained per subinterface.!--- All entries for all subinterface on Gig4/1 are seen here.bison# sh ips ip route int gig 4/1Codes: C - connected, S - staticNo default gatewayS 100.100.100.4/30 via 100.100.100.2, GigabitEthernet4/1.1000C 100.100.100.0/30 is directly connected, GigabitEthernet4/1.1000!--- IPS routes are derived from the main-interface, !--- not specifically per subinterface. The next hop is verified here.bison# sh cdp ne int gig 4/1Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-BridgeS - Switch, H - Host, I - IGMP, r - RepeaterDevice ID Local Intrfce Hldtme Capability Platform Port ID----------- ----- ------- ------ ---------- -------- -------TBM06033144 (Aspe Gig4/1 136 T S WS-C6506 3/7 !--- Use this command if the neighbor supports it to verify !--- physical connectivity.bison# sh fcip profile 1FCIP Profile 1Internet Address is 100.100.100.1 (interface GigabitEthernet4/1.1000)Listen Port is 3225TCP parametersSACK is enabledPMTU discovery is enabled, reset timeout is 3600 secKeep alive is 60 secMinimum retransmission timeout is 200 msMaximum number of re-transmissions is 4Send buffer size is 0 KBMaximum allowed bandwidth is 1000000 kbpsMinimum available bandwidth is 15000 kbpsEstimated round trip time is 1000 usecCongestion window monitoring is enabled, burst size is 10 KB !--- The profile parameters are an easy way to directly verify !--- the configured TCP/IP parameters per FCIP instance.bison# sh int fcip 1fcip1 is trunkingHardware is GigabitEthernetPort WWN is 20:c2:00:05:30:00:7a:dePeer port WWN is 20:42:00:0c:30:6c:24:40Admin port mode is auto, trunk mode is onPort mode is TEvsan is 1Trunk vsans (allowed active) (1,600-601)Trunk vsans (operational) (1,600-601)Trunk vsans (up) (1,600-601)Trunk vsans (isolated) ()Trunk vsans (initializing) ()Using Profile id 1 (interface GigabitEthernet4/1.1000)Peer InformationPeer Internet address is 100.100.100.6 and port is 3225Special Frame is disabledMaximum number of TCP connections is 2Time Stamp is disabledQOS control code point is 0QOS data code point is 0B-port mode disabledTCP Connection Information2 Active TCP connectionsControl connection: Local 100.100.100.1:64860, Remote 100.100.100.6:3225Data connection: Local 100.100.100.1:64862, Remote 100.100.100.6:32252 Attempts for active connections, 0 close of connectionsTCP ParametersPath MTU 3000 bytesCurrent retransmission timeout is 200 msRound trip time: Smoothed 2 ms, Variance: 1Advertized window: Current: 118 KB, Maximum: 118 KB, Scale: 1Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1Congestion window: Current: 10 KB, Slow start threshold: 112 KB5 minutes input rate 200 bits/sec, 25 bytes/sec, 0 frames/sec5 minutes output rate 200 bits/sec, 25 bytes/sec, 0 frames/sec1306 frames input, 891212 bytes472 Class F frames input, 46972 bytes834 Class 2/3 frames input, 844240 bytes0 Error frames timestamp error 0867 frames output, 252424 bytes470 Class F frames output, 48860 bytes397 Class 2/3 frames output, 203564 bytes0 Error frames 0 reass frames!--- Here, the specific details per FCIP interface are shown !--- by a running FCIP instance. You can also derive the !--- TCP parameters of the peer with this output.bison# sh fcdomain vsan 600The local switch is the Principal Switch.Local switch run time information:State: StableLocal switch WWN: 22:58:00:05:30:00:7a:dfRunning fabric name: 22:58:00:05:30:00:7a:dfRunning priority: 2Current domain ID: 0x01(1) Local switch configuration information:State: EnabledFCID persistence: DisabledAuto-reconfiguration: DisabledContiguous-allocation: DisabledConfigured fabric name: 20:01:00:05:30:00:28:dfConfigured priority: 128Configured domain ID: 0x01(1) (preferred)Principal switch run time information:Running priority: 2Interface Role RCF-reject---------------- ------------- ------------fcip1 Downstream Disabled---------------- ------------- ------------bison# sh fcdomain vsan 601The local switch is the Principal Switch.Local switch run time information:State: StableLocal switch WWN: 22:59:00:05:30:00:7a:dfRunning fabric name: 22:59:00:05:30:00:7a:dfRunning priority: 2Current domain ID: 0x01(1) Local switch configuration information:State: EnabledFCID persistence: DisabledAuto-reconfiguration: DisabledContiguous-allocation: DisabledConfigured fabric name: 20:01:00:05:30:00:28:dfConfigured priority: 128Configured domain ID: 0x01(1) (preferred)Principal switch run time information:Running priority: 2Interface Role RCF-reject---------------- ------------- ------------fcip1 Downstream Disabled---------------- ------------- ------------!--- Similar to normal (E)ISL troubleshooting, verify !--- that the fabric is formed as expected.bison# sh fcns da vsan 600-601VSAN 600:-------- ---- ----------------------- -------- ---------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE-------- ---- ----------------------- -------- ---------------------------0x010001 N 10:00:00:00:c9:32:a6:e3 (Emulex) scsi-fcp:init 0x020001 N 50:05:07:63:00:d0:94:4c (IBM) scsi-fcp:target fc..Total number of entries = 2VSAN 601:-------- ---- ----------------------- -------- ---------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE-------- ---- ----------------------- -------- ---------------------------0x010001 N 10:00:00:00:c9:32:a6:e2 (Emulex) scsi-fcp:init 0x010100 N 10:00:00:00:00:05:00:00 0x020100 N 10:00:00:00:00:01:00:00 Total number of entries = 3MDS 9216 (Canterbury)canterbury# sh int gig 2/1.500GigabitEthernet2/1.500 is up Hardware is GigabitEthernet, address is 0005.3000.ade6 Internet address is 100.100.100.6/30 MTU 3000 bytes 5 minutes input rate 2248 bits/sec, 281 bytes/sec, 0 frames/sec 5 minutes output rate 696 bits/sec, 87 bytes/sec, 0 frames/sec 2263 packets input, 482038 bytes 0 multicast frames, 0 compressed 0 input errors, 0 frame, 0 overrun 0 fifo 2479 packets output, 1077822 bytes, 0 underruns 0 output errors, 0 collisions, 0 fifo 0 carrier errorscanterbury# sh cdp ne int gig 2/1Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge S - Switch, H - Host, I - IGMP, r - RepeaterDevice ID Local Intrfce Hldtme Capability Platform Port ID----------- ----- ------- ------ ---------- -------- -------TBM06033144 (Aspe Gig2/1 165 T S WS-C6506 3/3!--- If the neighbor supports CDP, use this command!--- to verify physical connectivity.canterbury# sh fcip profile 1FCIP Profile 1Internet Address is 100.100.100.6 (interface GigabitEthernet2/1.500)Listen Port is 3225TCP parametersSACK is enabledPMTU discovery is enabled, reset timeout is 3600 secKeep alive is 60 secMinimum retransmission timeout is 200 msMaximum number of re-transmissions is 4Send buffer size is 0 KBMaximum allowed bandwidth is 1000000 kbpsMinimum available bandwidth is 15000 kbpsEstimated round trip time is 1000 usecCongestion window monitoring is enabled, burst size is 10 KBcanterbury# sh int fcip 1fcip1 is trunkingHardware is GigabitEthernetPort WWN is 20:42:00:0c:30:6c:24:40Peer port WWN is 20:c2:00:05:30:00:7a:deAdmin port mode is auto, trunk mode is autoPort mode is TEvsan is 1Trunk vsans (allowed active) (1,600-601)Trunk vsans (operational) (1,600-601)Trunk vsans (up) (1,600-601)Trunk vsans (isolated) ()Trunk vsans (initializing) ()Using Profile id 1 (interface GigabitEthernet2/1.500)Peer InformationPeer Internet address is 100.100.100.1 and port is 3225Special Frame is disabledMaximum number of TCP connections is 2Time Stamp is disabledQOS control code point is 0QOS data code point is 0B-port mode disabledTCP Connection Information2 Active TCP connectionsControl connection: Local 100.100.100.6:3225, Remote 100.100.100.1:64860Data connection: Local 100.100.100.6:3225, Remote 100.100.100.1:648620 Attempts for active connections, 0 close of connectionsTCP ParametersPath MTU 3000 bytesCurrent retransmission timeout is 200 msRound trip time: Smoothed 2 ms, Variance: 1Advertized window: Current: 118 KB, Maximum: 118 KB, Scale: 1Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1Congestion window: Current: 10 KB, Slow start threshold: 112 KB5 minutes input rate 184 bits/sec, 23 bytes/sec, 0 frames/sec5 minutes output rate 184 bits/sec, 23 bytes/sec, 0 frames/sec1163 frames input, 336700 bytes722 Class F frames input, 72176 bytes441 Class 2/3 frames input, 264524 bytes0 Error frames timestamp error 01588 frames output, 917216 bytes724 Class F frames output, 70288 bytes864 Class 2/3 frames output, 846928 bytes0 Error frames 0 reass frames

Note on Native VLAN Mismatch

Under some conditions, customers may see the native vlan mismatch error message in the system log of the adjacent (Catalyst) Ethernet switch. This indicates an incorrect configuration on the trunkport of those switches. If a subinterface of .1000 is configured on one of the main interfaces on the MDS IP Services module, the adjacent switchport on the Catalyst switch needs to be trunking for that same VLAN ID of 1000. If, for some reason, the Catalyst is running VLAN 1000 as Port VLAN or Native VLAN, then error messages are generated and connectivity is broken, because the dot1q encapsulated frames from the MDS side are not understood by the switch and will be discarded. An example of native vlan mismatch errors is shown below.