Let’s be honest…Microsoft owns you and always will…just when you think you are OUT…they pull YOU back IN!

This has become so much more complicated in this day and age. People think that mobile administrators are basically the peons from Warcraft. Do you remember the peons? I remember the peons! They think we’re these people! I mean c’mon!

Anyways, moving on back in the world of reality…mobility is the one area of IT that you cannot go to school for and you have the biggest challenges if you are truly an engineer and not some offshore help desk activating BlackBerrys (yes I went there). Many of these challenges that we face are RAMPANT in iOS apps that make our users hate us at times and make life so extremely difficult for us. Here are a few:

• Pretty much no centralized logging capabilities (don’t use the SDK excuse because that only saves you on internal apps and your MDM vendor apps)
• Ridiculous amounts of complexity from client to server
• By far the most difficult to debug. Call me when your desktop application is impacted by Verizon Wireless switching to IPv6.
• Fragmented versioning (which is why I love when people say oh Android O is the greatest thing since the birth of Steve Jobs and blah blah considering most devices that we are worried about won’t be able to upgrade to Android O anyways)
• Inability to control/regulate/or lock down the majority of devices
• Much much much more….I would need a 12 chapter book to cover all of the challenges we REALLY have. I hate to break it to the fine people at AirWatch, but NO WorkspaceONE cannot cure any of these things despite what some think.

I decided to focus this blog as your “Oh Shit Button” when something bad goes down with your office apps and what you can do. We will break it down into a few useful sections. First we’ll talk about the basics of an iOS 3rd party application and what matters to you, second we’ll talk about your tools, and finally we’ll bring them all together.

Office App Basics

So to talk about Office Apps, theres a few things you should know:

• App Data is containerized inside of the application. It’s like Object-Oriented Programming. An app and its data are together as one. It’s one of the major tenets of their security model on iOS, but it ALSO means that it is a major pain in the ass trying to diagnose issues as a support engineer.
• All iOS apps now use these things called App Groups and Keychain sharing to share credentials or key data that helps them work together. The best example is the AirWatch Agent and Boxer or Content Locker. They are part of a group or “family” that work together and can share stuff with each other.
• Office Apps on iOS now use Modern Authentication. What this means to you is #1 conditional access for non-Intune MDMs does not work anymore since it’s only supported for legacy authentication. That doesn’t count access policies that a Identity Manager product might have created. #2 it works on a two token system: (1) After authentication you are granted an access token that is good for a hour and (2) a refresh token is created which is good for 14 days, but with regular use can last up to 90 days. These tokens are the lifeblood of Office 365 authentication and without then you are screwed. You can read more bout that here. If necessary, this command will let you revoke a token for a user:Get-AzureADUser -SearchString towlesj | Revoke-AzureADUserAllRefreshToken

My Tool Box

So my tool box for working through these issues are mostly straight forward. They include:

• xCode for Device Logs
• iExplorer
• iOS Development Debugging Profiles
• Enabling Analytics/Full Diagnostics Logging

Some you might have heard of and others maybe not. So let’s get started…

xCode Device Logging

The logs give you some interesting information you can look at. Mostly you might be a nerd like me and find it interesting. This line below tells you what the process number is and that OneNote is allowed/entitled to run in the background:

CMSessionMgrHandleApplicationStateChange: CMSession: Client com.microsoft.onenote with pid ‘771’ is now Foreground Running. Background entitlement: YES

This one down here shows you its running in the foreground aka its being used and not in the background:

Sep 23 21:15:54 iPhone symptomsd(SymptomEvaluator)[178] <Notice>: 771 com.microsoft.onenote: ForegroundRunning (most elevated: ForegroundRunning)

Now it gets interesting, it lists the Team ID for the Microsoft App Group, shows you the ADAL version, and the results of the query for the Access/Refresh Tokens. It’s also setting up the cookie storage:

Sep 23 21:15:54 iPhone OneNote[771] <Notice>: ADAL 2.5.1 iOS 11.0 [2017-09-24 01:15:54] INFO: Using “SGGM6D27TK” Team ID for Keychain.
Sep 23 21:15:54 iPhone OneNote[771] <Notice>: ADAL 2.5.1 iOS 11.0 [2017-09-24 01:15:54] INFO: Found 31 token(s) for query: (resource <(null)> + client <(null)> + authority <(null)>)
Sep 23 21:15:54 iPhone OneNote(CFNetwork)[771] <Notice>: Faulting in NSHTTPCookieStorage singleton
Sep 23 21:15:54 iPhone OneNote(CFNetwork)[771] <Notice>: Faulting in CFHTTPCookieStorage singleton
Sep 23 21:15:54 iPhone OneNote(CFNetwork)[771] <Notice>: Creating default cookie storage with default identifier

One other interesting thing I like from the logs are these events where its staging a data report and putting it into a timed queue. Not necessarily useful, but it is informative.

Sep 23 21:15:57 iPhone OneNote[771] <Notice>: [ACT]:[SEMANTIC]:[Info]:Stage Receive: event name=reportdata, event priority=3, initId=EA5EF9ED-2F1C-4640-8258-57EAEA1B495B, timestamp=1506215757560
Sep 23 21:15:57 iPhone OneNote[771] <Notice>: [ACT]:[TIMEDQUEUE]:[Info]:1 event put into timed queue.
Sep 23 21:15:57 iPhone OneNote[771] <Notice>: [ACT]:[SEMANTIC]:[Info]:Stage Receive: event name=reportdata, event priority=3, initId=EA5EF9ED-2F1C-4640-8258-57EAEA1B495B, timestamp=1506215757561
Sep 23 21:15:57 iPhone OneNote[771] <Notice>: [ACT]:[TIMEDQUEUE]:[Info]:1 event put into timed queue.

You use these logs as the start of investigating an issue. You are looking for something basic here like token errors, exceptions, or anything that just looks “icky”

You should also use xCode to access crash logs as needed to provide to vendors or just look at the “hot mess” that it is.

You will find it in the Devices Section of xCode inside of the “View Device Logs” section

iExplorer

iExplorer becomes probably the best tool for anyone in the iOS support game. You can download it here, but do not listen to Microsoft support (the PC version is not even close being as good as the Mac one) and if your company doesn’t want to buy you a Mac Mini then perhaps you should look for greener pastures.

Basically the way that iExplorer works is you create a device backup and use iExplorer to traverse the backup. You can get access to “some” useful stuff. I use some lightly as it is not nearly as useful as you would like it to be, but beggars can’t be choosers.

You click Backup Explorer and let the fun begin

You want to remember to leverage the search and stick mainly to the App and App Group folders.

The hint that I will give you is that the files you are looking for are in these locations for an Office App:

• App > com.microsoft.onenote > Documents and grab all of the OneNote Logs and contents of the IntuneMAM folder (I’ll presume you have MAM policies since you are reading my blog).
• App Group > group.com.microsoft.onenote and all of the contents in there.
• Managed Preferences > Mobile (This is really cool. It actually shows you the managed app config that you push to your device. You can use this to confirm that you are pushing the proper UPN down to the device)

I know you are asking yourself, why is this useful? I’ll tell you why! Because I SAID SO! No really, with the app containerization it is the only way you can get these logs sometimes. If you have an app with a corrupted database, you might not be able to even launch the application. So what do you do then? You might have data that hasn’t sync’d or some other nightmarish situation. Let’s be honest these issues never happen to people in IT. They always happen to the last person you want it ot happen to.

iOS Development Debugging Profiles

Sometimes we need stronger and better logs. The xCode logs are a nice start, but sometimes we need more. You can find the profiles here but you will need a developer account to install the profiles.

These are the profiles that I would recommend if you are trying to work on an application issue:

• Background Networking (VERY helpful as it shows you logs for background processes)
• CFNetwork (Overall network logging)
• I strongly recommend Charles Proxy for essentially tethering your device through your Mac as a Proxy which gives you some amazing logs for debugging. This will let you connect your iOS device to an internal app without VPN as a great way to find root cause. It even lets you debug SSL traffic.
• TCP Dump is also a useful article which shows you how to take a TCP dump of your device. I’ll put the steps below as it’s probably one of my favorite utilities.
  • On a Mac run:rvictl -s {UUID}followed by the TCP Dump command:sudo tcpdump -i rvi0 -w DumpFile.pcap
• VPN

These tools will be very useful and typically can get you the logs that you need. There are so many aspects of this and they get more complex by the complexity of your environment. As an example, if you are using ADFS+VPN you should understand how it works. You will find that quite frequently with modern authentication that Microsoft apps suck at OnDemand VPN.

You absolutely MUST use Per App VPN! This is 2017 NOT 1997. If you are smart enough to use Cisco AnyConnect, they support Split Tunneling with Per App VPN, which makes sure that the applications you want to run VPN on actually run VPN. Otherwise, you have to hope that when it sees your ADFS server is unresolvable via mDNS responder that it will ACTUALLY fire VPN. I know this seems trivial, but you will see roughly once per year that OnDemand VPN is broken by your vendor…Thanks Pulse Secure for being Pulse Inconsistent.

Microsoft App Full Diagnostic Logging

I’ll be honest, it’s sort of shitty because you never actually enable full diagnostics logging in iOS until you have an issue. By that time, you are usually screwed.

You can enable this via Settings > OneNote > Help Us Improve and enable Full Diagnostic Logging.

Sorry my circling is an epic fail, but I’m an engineer not Rembrandt. You can enable the analytics capture, which is especially helpful for crash logs from Settings > Privacy > Analytics and enableShare with App Developers

You can actually go into the Analytics Data right above that section to find the IPS files for your various applications and airdrop them to whoever you need to.

Bringing it all together

So now that everyone is asleep from all of the information overload…let’s talk about bringing it all together. Not everything is useful for every situation, but one of the major things I preach is understand the client-server relationship. Focus on the issue at hand and use what makes sense!

If you are having issues with synchronization for example, you should be using network logs, background network logs, maybe even VPN logs along with a good old fashion packet capture. That is outside of having common sense and testing things. Remember the old tenet: “Is it plugged in?”

I think it is very important that as human beings that we don’t just do what sounds cool, but do what actually makes sense. Most of this is that people don’t work for true leaders. They just blindly follow the wrong people and trust those who are not trustworthy. I think as engineers we must think critically and be intelligent. I am someone who has a thirst for knowledge. I MUST know how things work and it MUST make sense to me. Sure I might be a bit of a “Sheldon” and I’m “NOT” for everybody, but I am smart enough to know that no one has all of the answers and we should learn something new everyday. I think the reason that people respect my blog and the things that I say is that I am pure.

I am not trying to sell someone, I don’t bullshit, or use “buzz words” like user adoption or one of the other things you probably heard at VMWorld/Ignite or anytime a C-level person speaks. It’s important to understand how technology works and that is why I love a healthy debate. Everyone has an opinion, and it’s up to you to determine if that opinion is stupid or not. I digress! I hope that this information is honestly helpful to you and your development as engineers because I believe in paying it forward every day.

–Jon