Sunday, January 29, 2017

Mobility Express on 3802i

Just got Cisco Aironet 3802i into my hands. Wanted to convert it into standalone aka autonomous mode... oh wait, it's gone. Mobility Express, this is what we have now.

Sounds quite charming, simply speaking you have Master AP which runs "WLC software" in addition to LWAPP software. It associates to itself al well as any other neighboring AP, so yes, if you have only one AP you can run both controller and access point functions on it.

Quick start guide and... my access point does not broadcast CiscoAirProvision SSID

On the console I see alternating messages:

[*06/15/2016 00:19:22.5103] CAPWAP State: Init
[*06/15/2016 00:19:22.5104] CAPWAP State: Discovery
[*06/15/2016 00:19:31.8579] CAPWAP State: Discovery
[*06/15/2016 00:19:31.8581] Did not get log server settings from DHCP.
[*06/15/2016 00:19:41.3609] CAPWAP State: Discovery
[*06/15/2016 00:19:41.3610] Did not get log server settings from DHCP.
[*06/15/2016 00:19:50.8633] CAPWAP State: Discovery

After research, this is what I need: Mobility Express User Guide 8.3 (previous versions do not include 3800 devices).

It says login to AP using Cisco/Cisco and do show version and look for "AP Image Type" and "AP Configuration" string in order to understand which version of software you use:


In my case - not present. Converting guide says to download tarball labeled "Software to be used for conversion from Lightweight Access Points only".

So I downloaded latest release for 3800:

The rest is also straightforward, get yourself tftp server, put your image there and use ap-type mobility-express tftp://ip/file command.

AP#ap-type mobility-express tftp://a.b.c.d/AIR-AP3800-K9-ME-8-3-102-0.tar
Starting the ME image download...
It may take a few minutes to finish download.
If it is longer, please abort command, check network connection and try again
########                                                                  11.6%[*06/15/2016 00:40:37.3653] CAPWAP State: Discovery
[*06/15/2016 00:40:37.3655] Did not get log server settings from DHCP.
###########################                                               38.8%[*06/15/2016 00:40:46.8688] CAPWAP State: Discovery
[*06/15/2016 00:40:46.8689] Did not get log server settings from DHCP.
###############################################                           66.4%[*06/15/2016 00:40:56.3713] CAPWAP State: Discovery
[*06/15/2016 00:40:56.3714] Did not get log server settings from DHCP.
###################################################################       93.6%[*06/15/2016 00:41:05.8742] CAPWAP State: Discovery
[*06/15/2016 00:41:05.8743] Did not get log server settings from DHCP.
#######################################################################  100.0%
Image downloaded, writing to flash...
do CHECK_ME, part1 is active part
tar: write error: No space left on device
sh: write error: No space left on device

Error, image file size is smaller than image+key+signature 0x0267e000 : 0x0283a839
Segmentation fault (core dumped)
part.bin signature verification failure, exit the script
Error: Image update failed.

Whoops! Again, ME User guide, quite handy:
After doing the same with .zip image, I got promising message on the console:

[*06/15/2016 00:51:38.8039] AP Type changed: CAPWAP to ME. AP Mode changed to flexconnect. AP Rebooting...

Access point went up with old software version. I try a couple of times more and I also upgraded lightweight software too:

AP#archive download-sw /reload tftp://a.b.c.d/ap3g3-k9w8-tar.153-3.JD.tar
(...)
do ACTIVATE, part1 is active part
activate part2, set BOOT to part2
AP primary version: 8.3.102.0
AP backup version: 8.2.20.16
Archive done.
Reload...

and then retry conversion to ME, both ways, without any luck.

AP#ap-type mobility-express tftp://a.b.c.d/AIR-AP3800-K9-ME-8-3-102-0.tar
Starting the ME image download...
It may take a few minutes to finish download.
(...)

Then during upgrades and reloads I spotted this errors:

Image downloaded, writing to flash...
do CHECK_ME, part2 is active part
tar: can't open '/tmp/part.tar': No such file or directory
No part.bin found
part to upgrade is part1
tar: can't open '/tmp/part.tar': No such file or directory
cp: can't stat '/bootpart/part1/info.ver': No such file or directory
activate part1, set BOOT to part1
grep: /tmp/info.ver: No such file or directory
AP primary version:
grep: /tmp/info.ver: No such file or directory
grep: /tmp/info.ver: No such file or directory
cp: can't stat '/tmp/part.tar': No such file or directory
Archive done.
[*07/29/2016 05:07:11.8915] AP Type changed: CAPWAP to ME. AP Mode changed to flexconnect. AP Rebooting...
[*07/29/2016 05:07:11.8966] AP Rebooting: Reset Request from Controller(AP Type Changed from CAPWAP to ME)

Writing reload timestamp (Fri Jul 29 05:07:11 UTC 2016) to disk

AP#[*07/29/2016 05:07:12.9426] umount: can't umount /storage: Device or resource busy
failed to unmount /storage; waiting 1s and trying again.
The system is going down NOW!05:07:13.9456]
Sent SIGTERM to all processes
[*07/29/2016 05:07:13.9512] device capwap0 left promiscuous mode
[*07/29/2016 05:07:13.9907] device capwap1 left promiscuous mode
Sent SIGKILL to all processes
Requesting system reboot83]
[07/29/2016 05:07:15.9400] reboot: Restarting system

And during booting:

UBIFS: mounted UBI device 0, volume 0, name "part1"
UBIFS: mounted read-only
UBIFS: file system size:   65646592 bytes (64108 KiB, 62 MiB, 517 LEBs)
UBIFS: journal size:       8888320 bytes (8680 KiB, 8 MiB, 70 LEBs)
UBIFS: media format:       w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root:  0 bytes (0 KiB)
part1/part.bin not found!
ubifsload - load file from an UBIFS filesystem

Usage:
ubifsload <addr> <filename> [bytes]
    - load file 'filename' to address 'addr'
Unable to load image part1/part.bin
Trying alternate UBIFS partition part2
UBIFS: mounted UBI device 0, volume 1, name "part2"
UBIFS: mounted read-only
UBIFS: file system size:   65646592 bytes (64108 KiB, 62 MiB, 517 LEBs)
UBIFS: journal size:       8888320 bytes (8680 KiB, 8 MiB, 70 LEBs)
UBIFS: media format:       w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root:  0 bytes (0 KiB)
Loading file 'part2/part.bin' to addr 0x02000000 with size 25215697 (0x0180c2d1)...
Done

Looks like we have two alternate boot partitions part1 and part2. Looks like installer is trying to install new image to part1 (it fails) while preserving image installed to part2 as a backup.

In case of emergency, it fail over to old image which is exactly what is happening to me.

So, I tried to set boot to part 1 once again manually and reload


AP#config boot path 1
AP#show boot
BOOT path-list:      part1
Console Baudrate:    9600
Enable Break:        yes
Manual Boot:         no
Memory Debug:        no
AP#reload
Proceed with reload? [confirm]

cli: AP Rebooting: CLI triggered reboot(reload command)

No luck, result was exactly the same as previously.
What if part1 is faulty indeed and we are stuck because installer see part2 as active and keep installing on part1? Maybe it can be cheated to think part1 is active and install to part2?
Risky, because in cause of failure we don't have backup but yea, what the heck.

THIS MAY CAUSE SERIOUS ISSUES WITH YOUR AP. YOU HAVE BEEN WARNED.

Interestingly enough, after I did this my installer still was smart enough to know that true active disk is part2, but at the same time it was able to successfully install image on part1 and boot from there.

AP#config boot path 1
AP#show boot
BOOT path-list:      part1
Console Baudrate:    9600
Enable Break:        yes
Manual Boot:         no
Memory Debug:        no
AP#ap-type mobility-express tftp://a.b.c.d/AIR-AP3800-K9-ME-8-3-102-0.tar
Starting the ME image download...
It may take a few minutes to finish download.
If it is longer, please abort command, check network connection and try again
#######################################################################   99.8%
Image downloaded, writing to flash...
do CHECK_ME, part2 is active part
Verify returns 0
part to upgrade is part1
activate part1, set BOOT to part1
AP primary version: 8.3.102.0
Archive done.
[*07/29/2016 05:20:14.9008] AP Type changed: CAPWAP to ME. AP Mode changed to flexconnect. AP Rebooting...
[*07/29/2016 05:20:14.9063] AP Rebooting: Reset Request from Controller(AP Type Changed from CAPWAP to ME)

(...)

UBIFS: mounted UBI device 0, volume 0, name "part1"
UBIFS: mounted read-only
UBIFS: file system size:   65646592 bytes (64108 KiB, 62 MiB, 517 LEBs)
UBIFS: journal size:       8888320 bytes (8680 KiB, 8 MiB, 70 LEBs)
UBIFS: media format:       w4/r0 (latest is w4/r0)
UBIFS: default compressor: LZO
UBIFS: reserved for root:  0 bytes (0 KiB)
Loading file 'part1/part.bin' to addr 0x02000000 with size 42182713 (0x0283a839)...
Done
SF: Detected N25Q32A with page size 64 KiB, total 4 MiB
Checking image signing.
This could take up to 40 seconds. Please wait...
Image signing verification success, continue to run...

(...)

Ok it works!

AP#show version | i MOBILITY
AP Image type    : MOBILITY EXPRESS IMAGE
AP Configuration : MOBILITY EXPRESS CAPABLE

After another series of console messages I can see confirmation that it work correctly:

Enabling Controller Provisioning
 Configuring management interface
 Enabling mgmt via wireless
 Enabling Provisioning SSID
 SSID: CiscoAirProvision, Admin Status: 1, Interface Name: management, 802.11 Auth: WPA2-PSK,  Wi-Fi Protected Access : Enabled

So I can jump into this SSID and carry on with configuration from the GUI.

Connect to CiscoAirProvision SSID using key "password", go to http://192.168.1.1 and very nice wizard will take you from there.

Edit 16.03.2017:
I had a lot of issues with WiFi after upgrade from 3700 to 3800. Including some devices failed to connect, some failed after couple of days and AP restart was required, RF Optimization had to be turned of because of similar reasons. It was so serious that I almost decided to get back to 3700. At the end of the day I found out that there's new software version for that AP, previously there was only one software available. I'm still keeping my fingers crossed, but looks like after upgrade to AIR-AP3800-K9-ME-8-3-111-0 everything is all right.