How to Fix: Cisco Mobility Express Controller (Access Point) keep disconnecting/excluding users/clients

The issue

Cisco Mobility Express Access Point keep disconnecting/excluding clients/users from time to time, the configuration seems fine on the controller but somehow, it keeps excluding clients.

Sometime we can discover following errors

[Date] [Time] [AP IP address] [AP Name]: *Dot1x_NW_MsgTask_0: [Date] [Time]: %DOT1X-4-MAX_EAP_RETRIES: 1x_auth_pae.c:6710 Max EAP identity request retries (3) exceeded for client [MAC Address]

The above error can be caused by many factors, some of them can be, low signal strength between AP and the client, RF interference etc. which cause the client to keep re-authenticating, eventually caused a behavior seemed with many tries of authentication, this triggers the protection from “Client Exclusion Policy” finally the client gets excluded for a period of time. The results will be the clients keeps getting disconnected. Local EAP parameters can be one of the reason as well.

The Fix

Distribute the access points correctly at right distance, adjust antenna power for access points correctly, configure the RF frequency correctly to minimize interference, eventually improve the RF signal quality, and strength reaching out at clients. That should reduce the error.

Workaround

There are some workaround may or may not work

(Cisco Access Point disconnecting clients from time to time can be caused by signal issue plus following settings, we can use following workaround to get around with it but it’s not recommended to disable them completely for enterprise environment since those are security features.)

1 If you get a lot of excluded clients try to follow this “How to: Check/Enable/Disable Cisco Controller (Access Point) Client Exclusion Policy settings (Mobility Express) via Controller Console” to disable “Client Exclusion Policies”. So that they will not be excluded. (Note: This is a security feature, we really should fix the root cause rather than disabling Client Exclusion Policies, especially within enterprise environment)

2 If you are getting a lot of similar errors in red at the top of this page, try to follow this “How to Check/Change: Cisco Controller/Mobility Express (Access Point) Local EAP settings, commands” to increase value for “EAP-Identity-Request Max Retries” available value is 1 to 20, Recommendations for the Max Retries is 12.

More information about EAP-* parameters can be found in “How to Check/Change: Cisco Controller/Mobility Express (Access Point) Local EAP settings, commands


How to Check/Change: Cisco Controller/Mobility Express (Access Point) Local EAP settings, commands

EAP-Identity-Request Timeout (seconds)
EAP-Identity-Request Max Retries
EAP Key-Index for Dynamic WEP
EAP Max-Login Ignore Identity Response
EAP-Request Timeout (seconds)
EAP-Request Max Retries
EAPOL-Key Timeout (milliseconds)
EAPOL-Key Max Retries
EAP-Broadcast Key Interval
RSN Capability Validation

Show current Local EAP settings

1 Login to Cisco Controller (Mobility Express) via console or SSH

2 Type following command

show advanced eap
show advanced eap
show advanced eap

Change Local EAP settings

config advanced eap [name] [value]
config advanced eap ?
config advanced eap ?

Bonus

Increase value for “EAP-Identity-Request Max Retries” may fix or reduce following error

[Date] [Time] [AP IP address] [AP Name]: *Dot1x_NW_MsgTask_0: [Date] [Time]: %DOT1X-4-MAX_EAP_RETRIES: 1x_auth_pae.c:6710 Max EAP identity request retries (3) exceeded for client [Client MAC Address]

More information about EAP-* (non-Cisco official)

EAP-Identity-Request Timeout:

This timer affects how long we wait between EAP Identity  Requests.  By default this is one second (4.1 and lower) and 30 seconds  (4.2 and greater.  The reason for this change was, we found that some  clients, hand helds, phones, scanners etc, had a hard time responding  fast enough.  Devices like laptops, usually do not require a  manipulation of these values.  Available value is from 1 to 120.

So, what happens with this attribute set to a value of 30?  When  the client first connects, it sends and EAPOL Start to the network, the  WLC sends down an EAP packet, requesting the user or machines Identity.   If the WLC does not receive the Identity Response, it sends another  Identity Request 30 seconds after the first.  This happens on initial  connection, and when the client roams.

What happens when we increase this timer?  If everything is good,  there is no impact.  However, if there is an issue in the network  (including client issues, AP issues, RF issues), this can cause delays  in network connectivity.  For example, if you set the timer to the  maximum value of 120 seconds, the WLC waits 2 minutes between Identity  Requests.  If the client is roaming, and the Response is not received by  the WLC, we have created, at minimum, a two minute outage for this  client.

Recommendations for this timer is to set it at 5.  There is no  current reason, to place this timer at it’s maximum value.

EAP-Identity-Request Max Retries

So, for max retries, what does this value do?  In short, this is  the number of times the WLC will send the Identity Request to the  client, before removing it’s entry from the MSCB.  Once the Max Retries  is reached, the WLC sends a de-authentication frame to the client,  forcing them to restart the EAP process.  Available value is 1 to 20.   So let’s look at this for a moment.

The Max Retries is going to work with the Identity Timeout.  If  you have your Identity Timeout set to 120, and your Max Retries to 20  how long does it take for the client to be removed?  120 * 20 = 2400.   So it would take 40 minutes for the client to be removed, and to start  the EAP process over again.  If instead you set the Identity timeout to  5, with the Max Retires of 12, 5 * 12 = 60.  So there is one minute  until the client is removed, and it has to start EAP over.

Recommendations for the Max Retries is 12.

EAPOL-Key Timeout

For the EAPOL-Key Timeout value, the default is 1 second or 1000  milliseconds.  What this means is when it comes time to exchange the  EAPOL keys between the AP and client, the AP will send the key and wait  up to 1 second by default for the client to respond.  After waiting the  defined time value, the AP will re-transmit the key again.  You can use  the command “config advanced eap eapol-key-timeout <time>” to alter this setting.  The available values in 6.0 are between 200 and  5000 milliseconds, while codes prior to 6.0 allow for values between 1  and 5 seconds.  Keep in mind that if you have a client which isn’t  responding to a key attempt, extending the timers out can give them a  little more time to respond….however, this could also prolong the time  it takes for the WLC/AP to deauthenticate the client in order for the  whole 802.1x process to start fresh.

EAPOL-Key Max Retries

For the EAPOL-Key Max Retries value, the default is 2.  What this  means is that we will retry the original key attempt to the client 2  times.  This setting can be altered using the command “config  advanced eap eapol-key-retries <retries>”.  The available  values are between 0 and 4 retries.  Using the default value for the  eapol key timeout (1 sec) and the default value for the eapol key retry  (2) the process would go as follows if a client doesn’t respond to the  initial key attempt:

1 – AP sends key attempt to the client
2 – Wait 1 second for a reply
3 – If no reply, then send eapol key retry attempt #1
4 – Wait 1 second for a reply
5 – If no reply, then send eapol key retry attempt #2
6 – If there is still not a response from the client and the retry value  is met, then deauthenticate the client.

Again, as with the EAPOL-Key Timeout, extending the EAPOL-Key  retry value could in some circumstances be beneficial, however setting  it to the max may again be harmful as the deauthenticate message would  be prolonged. [2]

Resources

[1] Information About Local EAP
[2] EAP Timers on Wireless Lan Controllers


How to: Check/Enable/Disable Cisco Controller (Access Point) Client Exclusion Policy settings (Mobility Express) via Controller Console

Types of Client Exclusion Policies for Mobility Express Controller

Excessive 802.11 Association Failures—Clients are excluded on the sixth 802.11 association attempt, after five consecutive failures.
Excessive 802.11 Authentication Failures—Clients are excluded on the sixth 802.11 authentication attempt, after five consecutive failures.
Excessive 802.1X Authentication Failures—Clients are excluded on the fourth 802.1X authentication attempt, after three consecutive failures.
IP Theft or IP Reuse—Clients are excluded if the IP address is already assigned to another device.
Excessive Web Authentication Failures—Clients are excluded on the fourth web authentication attempt, after three consecutive failures. [1]

How to: Check current Client Exclusion Policy settings for Mobility Express Controller

1 Connect to the Mobility Express controller via console or SSH

Cisco Mobility Express Controller via SSH
Cisco Mobility Express Controller via SSH

(Use “show exclusionlist” to check excluded users )

show exclusionlist
show exclusionlist
show exclusionlist

2 Use “show wps summary” to show current Client Exclusion Policies

show wps summary
show wps summary
show wps summary

How to: Modify/Change Client Exclusion Policies for Mobility Express Controller

To disable “Excessive 802.11-association failures”

config wps client-exclusion 802.11-assoc disable

To enable “Excessive 802.11-association failures”

config wps client-exclusion 802.11-assoc enable

To enable/disable Excessive 802.11-authentication failures

config wps client-exclusion 802.11-auth {enable | disable}

To enable/disable Excessive 802.1x-authentication

config wps client-exclusion 802.1x-auth {enable | disable}

To enable/disable IP-theft

config wps client-exclusion ip-theft {enable | disable}

To enable/disable Excessive Web authentication failure

config wps client-exclusion web-auth {enable | disable}

Resources

Cisco Wireless LAN Controller Configuration Guide, Release 7.4 [1]


How to: Update Cisco Mobility Express with Web GUI and tftp without using terminal

1 Download correct firmware for the device

2 Download a tftp server

Click here to download SolarWinds TFTP Server for free

3 Navigate to “Management -> Software Update”

4 Fill correct information

Transfer ModeTFTP
IP Address(IPv4)/Name *Your tftp server IP address
File Path */
Auto RestartCheck
Cisco Mobility Express -> Management -> Softwar e Update
Cisco Mobility Express -> Management -> Softwar e Update

5 Extract Cisco firmware “AIR-APxxxx-Xx-ME-8-10-112-0.zip” packet to a folder, in the example we use “D:\cisco”

We should have following folder structure

D:\cisco\ap_supp_list.inc
D:\cisco\ap1g1
D:\cisco\ap1g4-capwap
.....
D:\cisco\apname_decoder.inc
.....
D:\cisco\version.info
 
etc. etc.

6 Launch SolarWinds TFTP Server or your favourite tftp server software

7 Configure the tftp server, in the example we use SolarWinds TFTP Server, Click on “File”

SolarWinds TFTP Server
SolarWinds TFTP Server

8 Then click on “Configure”

9 Make sure we have entered the correct “TFTP Server Root Directory” then click on “Start” button to start the TFTP service, finally click on “OK” button

9.1 Make sure you have allowed incoming connection to the tftp server in the windows firewall (In other words, make sure the access point can successfully download the files from the tftp server successfully)

10 Back to Cisco Mobility Express, double check the details filled are all correct, click on “Update” button to begin the Update process. (Update progress information will be displayed at the top of the page)

(Do not disconnect the power/Ethernet cable from the access point, or stop the tftp service etc. It may damage the device)

(You should be able to see the log from the tftp server)

SolarWinds TFTP Server Log
SolarWinds TFTP Server Log

11 Once it’s done the device will restart if you have checked “Auto Restart”

12 When fully booted, you should be able to use it again.

(Don’t forget to stop the TFTP service)


Cisco Mobility Express – Use web GUI to switch Primary image/Backup image (How to swap primary image/backup image via web gui)

(Click here to read: Cisco Access point/Switch swap primary boot image/backup boot image via command line/terminal)

1 Login to your Cisco Aironet xxxx Series Mobility Express or Cisco Mobility Express web GUI

2 Navigate to “Monitoring -> Network Summary -> Access Points”

Cisco Aironet xxxx Series Mobility Express -> Monitoring -> Network Summary -> Access Points
Cisco Aironet xxxx Series Mobility Express -> Monitoring -> Network Summary -> Access Points

3 Find “TOOLS” Tab and click on it

Cisco Aironet xxxx Series Mobility Express -> Monitoring -> Network Summary -> Access Points -> TOOLS
Cisco Aironet xxxx Series Mobility Express -> Monitoring -> Network Summary -> Access Points -> TOOLS

4 Save it by clicking on the save button on the top right corner

5 Restart the Access Point

Navigate to “Advanced -> Controller Tools -> Restart Controller -> Click on “Restart Controller” button”

6 Once it’s back online, it should be running with swapped image


Cisco Access point/Switch swap primary boot image/backup boot image

Keywords: Cisco Boot Image, Cisco access point, access point, primary boot image, backup boot image, ap running image, cisco mobility express, switch primary boot image, switch backup boot image, swap primary boot image, swap backup boot image

(Click here to read: Cisco Mobility Express – Use web GUI to switch Primary image/Backup image)

1 Connect to the device via console/SSH (If using Mobility express, connect to the device IP via SSH or console cable, not the Mobility Express controller IP address)

2 Login with correct credential

3 You will see “name of the device> ” in command window

Console command window
Console command window

4 Type “show version” to view current image information

Console command - show version
Console command – show version

5 From following image we can see currently the AP us running on “Primary Boot Image” which is “8.8.120.0” the “Backup Boot Image” is “8.10.105.0”

AP Running Image, Primary Boot Image, Backup Boot Image
AP Running Image, Primary Boot Image, Backup Boot Image

6 Now we need to login to “Cisco Controller”. For Mobility Express running as master or “Autonomous Mode” you can SSH to the IP address which you used to login to Mobility Express controller (See following image).

Cisco Mobility Express Controller
Cisco Mobility Express Controller
SSH -> Cisco Mobility Express
SSH -> Cisco Mobility Express

7 Enter following command, replce <AP_Name> with correct access point name

(Cisco Controller) >config ap image swap <AP_Name>
config ap image swap <AP_Name>
config ap image swap MyAP
config ap image swap MyAP

8 If there is no error message returned, reboot the AP.

After the AP rebooted, use the same “show version” command to check the booted image, now it should be swapped. (Or use Wireless LAN Controller Web UI to check current version number)

Cisco Aironet activate Mobility Express Mode (Switch from CAPWAP to Mobility Express Mode)

To switch from CAPWAP (Control And Provisioning of Wireless Access Points) mode to ME (Mobility Express) mode

1 Connect PC and Access Point with console cable.

2 Open Device Manager

3 Under”Ports”, find out what port is used

4 Select “Serial” and put correct port into Putty

Putty - Serial, Serial line (Port)
Putty – Serial, Serial line (Port)

5 Click on Open button from Putty

6 Default Username: “Cisco” Password:”Cisco” for console

7 Follow the prompt to change the default password and finish initial configuration or autoinstallation

8 Type enable then hit Enter key and execute following command

AP#show version

If it shows:

AP Image Type: MOBILITY EXPRESS IMAGE
AP Configuration: MOBILITY EXPRESS CAPABLE

Then continue with:

AP#ap-type mobility-express

(Note: If you are not runing Mobility express image you will need to download the correct image and then setup TFTP server then use following command to update the system)

AP#ap-type mobility-express tftp://<TFTP Server IP>/<path to tar file>

9 Wait for 10 minutes

10 Power cycle the Access Point and wait for 20 minutes.

11 Find out the IP address of the Access Point

12 Access the address from browser with the credential you have set from step 7

Now you have access to Mobility Express (See following image) rather than the simple CAPWAP page.

Cisco Mobility Express Login Page (Wireless LAN Controller)
Cisco Mobility Express Login Page (Wireless LAN Controller)

How to redirect printer, Transfer printer queue to another printer

If you have a long print queue, but the current default printer is not working, use this trick to redirect the current and future print queue to another working printer.

1 Use key combination Win + R to open run window

2 Enter control printers, press enter

3 Devices and Printers window will show up

4 Right click on the dead printer -> Printer properties -> Ports -> Select another printer which works -> Click on OK button

From now on, the old print queue and future print queue will be processed form the selected new printer, if in a domain setup, it can save you from changing all default printers for all affected users by doing this from the server side.

How to: Use VNC to remote control printers or multifunction machine

http://files.sharpusa.com/Downloads/ForBusiness/DocumentSystems/MFPsPrinters/Manuals/MX2640_3140_3640/contents/01-040.htm (Not Working anymore)

https://www.realvnc.com/en/connect/download/vnc/

Some multifunction machine or printers allow remote control of operation panel.

To have this ability you need to change some settings to enable this function.


For example: Sharp MX-4070N.

1 Login to configuration page.

2 Navigate to: System Settings > Common Settings > Operation Settings > Condition Settings > Remote Operation Settings

3 Change “Operational Authority:” to “Allowed”

4 Download VNC from official website: https://www.realvnc.com/en/connect/download/vnc/

5 Launch the VNC application

6 Enter IP address into top address bar e.g. 10.0.0.2

7 Press Enter to connect (You might need to permit access physically at multifunction machine operation panel)

Now you should be able to see identical screen from operation panel and able to operate from VNC

Printer printout cut off / incomplete issue

Sometimes printout can have cutoff issue, which seems like hardware problem, usually vertical or horizontally cutting off.

Make sure the size settings is right from software e.g. Word, printer driver etc.

Make sure paper is placed properly.

Make sure you are using the latest driver for the printer.

If the problem still persists, try to uninstall the driver and reinstall, but using PostScript (or PS) instead of PCL driver if the printer supports it (Some printers only support PCL). It might fix the cutting off issue.

Note: It is always an good idea to download driver and software from official website.

If the issue is still not fixed, the hardware may have some problems.