Tips for building a scalable enterprise deployment of Tableau (Tableau Conference 2018)

Here’s my talk from Tableau Conference 2018 in New Orleans.

Check out the presentation here

UBS | Tips for building a scalable enterprise deployment of Tableau

So, you’ve deployed Tableau and have 100 users? Are you ready for that 100 to become 1,000 users? What about 10,000? Tableau is a great tool and can spread like a virus. Join this session, led by Paul—who’s grown his Tableau Centre of Excellence from zero to 13,000 users, while maintaining a solid, affordable deployment with an engaged user base and dynamic community. Learn top tips for scaling your service, such as: scaling your infrastructure, monitoring performance, capacity, organizing your team and support, and more.

Speaker(s):
Paul Banoub, UBS
Content Type: Breakout Session
Level: Intermediate
Track: Enablement and Adoption

Our Tableau centre of excellence: Managing an Enterprise deployment of Tableau (Tableau Conference 2017)

Hi – been asked for this video a few times. It’s my talk from Tableau Conference 2017.

Check out the presentation here

UBS: Our Tableau centre of excellence: Managing an Enterprise deployment of Tableau

At UBS we run a global Tableau Centre of Excellence, supporting 10k users across the business and IT. Over the last 3 years we have built a reputation as one of the most dynamic, well-run and user-focused IT services in the firm. In this session you will learn how we:
– Implemented and rolled out the infrastructure & application
– Manage upgrades & demand
– Run multiple training programs for users & execs
– Monitor performance, availability & capacity
– Run a dynamic, fun community of Tableau enthusiasts
– Work with the vendor to contribute to product evolution

This session will provide you with many practical tips and tricks to take back to your own organisations to enhance your deployments of Tableau.

This is part of the financial services track.

Speaker:
Paul Banoub, UBS
Content Type: Breakout
Level: Intermediate
Track: Enablement & Adoption
Tags: Financial Services

How To Set Up Your Tableau Server Environments

Hi,

Guess what this post is about – yes TABLE CALCULATIONS…. haha. No chance. Talk to Jonathan Drummey about those. This is of course yet more info that I hope will help you guys set up a dream Enterprise Tableau deployment.

Today we are gonna talk about Environments – i.e. what Tableau environments should you create in your organisation to give your team the best chance of success and keep your lovely users happy?

As always, I’m not saying this is THE way to do it. There are tons of great setups out there. I’ll just tell you what we have. Feel free to suggest better methods in the comments.

 

Environments for your users

This section is concerned with environments that you will provide for your Tableau users to do their work. Typically this will follow the standard Information Technology Infrastructure Library (ITIL) environment definitions, but there are a few things you can do to add extra options for your users.

These are the environments our users have at their disposal:

  • Production – The main business & user facing environment. Content published here is authoritative, follows best practice (hopefully) and is actively supported.
  • Testing – aka UAT. Generally used for final testing of uploaded content
  • Development – The environment where content is first shared as part of the development process.
  • Scratch – An extra environment for content that doesn’t need environment management. E.g. User wants to temporarily share content with a couple of colleagues.

Providing these environments gives users crucial options and flexibility. Your Tableau service will most likely serve many different business areas and teams, each with different practices for content development and release management. Some teams will rigorously follow Systems Development Lifecycle (SDLC) processes, creating content in development, promoting to User Acceptance Testing (UAT) and then eventually to Production. Other teams are totally happy to change content directly in Production, as and when they feel like it.

Crucially we don’t mandate what our users do, it’s a self-service model and so long as they follow their own due-diligence and governance procedures then that’s cool with me. The important thing is that we give them options to work with Tableau in the way that they want. If they break anything then they know it’s down to them.

The scratch environment is an interesting concept. It started with good intentions but realistically not many people are using it. So it looks like we might bin that.

Note that we use Tableau sites to segregate our environments.

 

Environments for your team

This is different from the above user-facing environments. These are the environments that your team uses for the service you provide. Obviously all this costs money in terms of hardware procurement and usage, depending on the spec you choose.

  • Production – Main environment that serves your users. In our environment this also includes the UAT, Development & Scratch sites for users – but we class it all as production. That might seem odd, but remember that many teams will be development teams, and to them the development site / area is their equivalent of production. So if the development site is down then they can’t work.
  • Disaster Recovery (DR) – For use in the event of a Production outage that can’t be easily restored. Exact same spec as Production. Totally identical, so that config can be restored and this server can be used as Production. You’ll need to make sure this environment gets the same upgrades as your Production environment.
  • UAT – This is UAT for my team. If we want to make a change to Production, it gets final testing here. This environment is also the exact same spec as Production to ensure an accurate test. If it fails here then it’s likely to fail in Production as well. We use UAT for testing maintenance releases, config changes and other potentially disruptive non-Tableau related changes to the server. Additionally, we make this environment available to users for a couple of weeks UAT prior to releasing new versions to production.
  • Engineering – Lower spec than prod & UAT. For testing the latest available release from Tableau. That is likely to be a higher version than production. Is useful for spotting bugs in new versions or confirming that bug-fixes work.
  • Beta Test – We are proud to be part of Tableau’s pre-release testing audience. We use this server to test releases in the Beta programme. Lower spec than engineering. To the point that the server only just meets the minimum requirements.
  • Alpha Test – We use this to test the alpha releases or any extra work we may be doing with developers at Tableau. We love to be involved in the genesis of new functionality.

So that’s what we are lucky enough to have. It’s not perfect but it allows us to give our users a ton of flexibility in how they use Tableau, and also my own team always has a place to test new releases, plan upgrades and help Tableau with their pre-release programmes.

Interested to see what the community has in terms of environments. Let me know in the comments. Remember there are a load of other posts on this blog about Enterprise Tableau considerations.

Cheers, Paul

How To Train Your Tableau Users

Hello there,

More tips coming to help you build that dream Tableau Server setup in your global Enterprise…. This time we focus on training.

Training is a critical subject. It won’t take long before someone asks you about your strategy or what you offer so you’ll need to ensure you have a professional sounding answer. Here’s what I offer my users. Hopefully some of this will be useful.

Screen Shot 2016-01-06 at 16.46.07

Training gets pride of place on our community site

General Considerations

Before I get into the specific offerings there are a few things you need to consider.

 

Branding is Critical!

So before I describe what we offer, you’ll notice that we have snappy names for each of these offerings. It’s not just “Tableau Training”. Much of Tableau’s success is down to stellar marketing and branding – take “Tableau Dr.” for example.

So make sure you think about what you are offering, how it will be perceived and how you can maximise adoption. A memorable and consistent name is key here. Each offering also has a nice-looking one pager in our Sharepoint slide library and an appropriate user-facing page / description on our community site so that we can give people all the info at a moments notice. The more effort you put into branding the better things will be.

You’ll also find that any of these offerings are easily transferable to teams / services that may be related to your area so there are a ton of collaboration opportunities here.

 

Consider the Timezones

If you’re like me then you’ll have users all over the globe. That means timezone aggro. You’ll generally need to double up most of your offerings. We are based in London so we have an early session for Asia/Pacific and an afternoon session for Europe/Africa/USA. Users will really appreciate this extra effort.

 

Don’t Worry About Attendance

It’s important not to get fixated on how many people are attending your sessions. Sometimes we get 3 people, sometimes we get dozens. Don’t worry about it. Training 3 people is better than none. And often the smaller sessions have better engagement.

 

Manage the Schedule, Don’t Let it Manage You!

You’re gonna get a LOT of requests for training. Make sure you control the schedule. Don’t be scared to tell users when the next scheduled session is and that they can join it. Don’t be scheduling things on demand of users or you’ll lose control completely. If training is regular and consistent then users will settle into a pattern and you’ll be able to manage your team’s time much more efficiently.

 

Track your Results

Training gets a lot of focus with senior management. You (or your manager) will get asked plenty about how many people you’re training and from what business area / region they are.

As we are all data people it’s much better to SHOW the execs the data rather than tell them you’re just “doing loads of training”. We have several Tableau vizzes that track the attendance at each of our training offerings. We then blend that with staff / hierarchy data to allow detailed reporting on all of the modules. That makes a much better impression.

For example just checking our viz now I can see that my team has done 545 Tableau Dr. Sessions this year, covering 277 different individuals. That’s almost 550 man hours of training on Dr. Sessions alone. Right there, evidenced and visualised. That makes a much bigger impression with the folks at the top.

unnamed

Tracking our training…

Don’t Get Lazy!

All this training can be a real time-burner. I’m sure some of you are thinking why don’t you just record the sessions and chuck the video online. And we get asked that. But never under-estimate the value of Instructor-Led sessions over videos. Our sessions are interactive, dynamic, enjoyable and make the users feel valued. They can be funny, and go in different directions according to the particular vibe in play. So don’t cave in to laziness, make sure the sessions happen and that they have that human touch.

So those are the general things to be aware of. I’ll now describe the specific options we have for training at my organisation.

 

Specific Training Offerings

Tableau Self-Learning Pipeline

I love self learning. Getting stuck into a manual, book or video. Or just firing up Tableau and seeing where it takes me. And you’ll get a lot of users that are the same. It’s always great to allow the self-learners to flourish, and it has the added benefit of not using your team resources.

You’ll get asked these questions hundreds of times, so make sure that you have all the relevant materials easily accessible on your community page.

  • What is Tableau?
  • How do I get Tableau?
  • How much does Tableau cost?
  • How can I get started with Tableau?

Our 101 page is top of the Community site in a super easy-to-find location. We refer our users to

This is generally more than enough for your average self-learner to get stuck into.

 

Create a Training Hierarchy

Users operate at different levels of ability and interest. So it is important that your training caters for that. It also looks great if you can demonstrate an end to end understanding of training. In addition to the self-learning pipeline, we offer the following.

 

Tableau Desktop Training Syllabus

This is my main training programme for Tableau Desktop users. It consists of 7 modules, each conducted once a month, with a session in the morning to hit APAC users and one in the afternoon to cover EMEA/USA users.

  • Module 1 – Introduction to Tableau
  • Module 2 – Data Visualisation
  • Module 3 – Table Calculations
  • Module 4 – Blending & Joining
  • Module 5 – Creating Effective Dashboards
  • Module 6 – Performance & Troubleshooting
  • Module 7 – Using Tableau at THIS ORGANISATION
Screen Shot 2016-01-06 at 16.41.58

Our training syllabus

Content is self-explanatory from the headings, but module 7 may seem odd. This is actually a pretty important consideration in large organisations where no two Tableau deployments will be the same. There will be organisation-specific nuances or considerations in many aspects of using Tableau and our place is no different. Much of this focuses on purchasing / onboarding, getting started, and change / incident management procedures, often a source of confusion for users.

Tableau Dr. Sessions

images

The Doc is in!

Ok so you all know these. The famous one-on-one consultancy time that Tableau offer at conferences. That’s all this really is, but we find that our users love the dedicated time to discuss whatever Tableau related topics they want. Many often come back for repeated sessions and some are almost data hypochondriacs!

The Dr. Sessions have been a real success. People at high-pressure organisations like Investment Banks, especially senior folk, really understand and appreciate the importance of that dedicated, individual, focused consultancy time. Time is the most precious commodity in such an organisation and you’ll find that you get some serious kudos for these.

When you’re at conference and see people gagging for Tableau Dr. Sessions, imagine what that would be like at your own organisation. Super-cool aint it?

 

Tableau In Focus

This offering is really cool. These are monthly Webinar sessions, again one for APAC timezone and one for EMEA/USA timezone. They are conducted by experts from Tableau, tailored to UBS requirements.

Subjects have included

  • Five Ways to Improve Dashboard Performance
  • Data Blending & Joining
  • Table Calculations
  • Deep Dive into LOD Expressions
  • Advanced Mapping
Screen Shot 2016-01-06 at 16.43.15

Tableau In Focus

Again, the fact Tableau are prepared to conduct these for us not only gives us valuable extra learning offerings but makes my users feel even more special than I know they are! We get great feedback, with users thrilled that we have such a good relationship with Tableau. That all gives them confidence in our service, and that is critical.

 

Tableau Executive Track

OK so this is an interesting one. We get a lot of senior folk asking for Tableau training. That’s very encouraging, obviously, but your average MD isn’t gonna sit through an hour Webex on LOD calcs.

So – what we do is our “Tableau Executive Track”. That’s a 1hr session, ideally in person with my laptop where we plan to cover the following.

  • Overview of the Tableau Service
  • Introduction to the Tableau Server
  • Showcase of current use across the business
  • Demo of custom admin views & introspection
  • How to build basic views in Desktop

In reality we often don’t get 30 seconds into this agenda before it is taken somewhere else, at the subject’s discretion. That’s cool – it’s what senior folk do and you need to be prepared for it. But it’s also important to come armed with an agenda to show you have a plan. Don’t just rock up and say “whadda you wanna know?” – you’ll get eaten alive.

Screen Shot 2016-01-06 at 16.43.24

Tableau Sr. Management Training

This is a really popular training offering, with many seniors having attended or planning to attend. It shows we are putting a great deal of thought into our training and that we are flexible enough to adapt to the variety of users we have.

 

Content Specific Training

Sometimes we get asked, “I’ve been sent this dashboard but I have no idea how to use it”. Well that’s NOT something we assist with. Our service is totally self-serve and it’s the responsibility of the dashboard author to

  • Ensure they have followed best practice
  • Include clear instructions for dashboard use or a link to a page that has such information
  • Ensure that their audience is briefed and aware of the dashboard usage

This is NOT down to us. We spend a lot of time policing content to make sure that authors are mindful of this as a dashboard that is hard to understand can adversely affect perception of the tool / service as a whole.

 

Wrap Up

So that’s what we offer. Aint it a lot? And boy is it important to the success of your service. Training can make or break a service and between you, me and the whole Internet – a lack of training can be used as an excuse for not engaging, or not performing. It’s rare but it happens.

Training your users should be enjoyable. If you find that it is becoming a pain then take a step back, consult some of your power users and modify your offerings.

In the time I was proof-reading this post the excellent Carl Allchin, who implemented much of our training when on site with us, posted some of his own thoughts on the Information Lab blog. Check them out. Also a big thanks to Andy Pick who has delivered dozens of sessions for our users.

As always I’m happy to jump on a call and discuss any of this. Now if you’ll excuse me I have a training session to deliver…

Cheers, Paul

How to PROPERLY Back Up Your Tableau Server

Hello there,

Whadda ya mean you didn't take a backup?

Whadda ya mean you didn’t take a backup?

Time for another post about Tableau Server and how to get the best out of it in a large-scale, enterprise deployment situation.

Today we are focusing on how to PROPERLY back up your Tableau Server installation.

Like many aspects of enterprise services, this is a simple concept, but one that if you get wrong, can spell disaster. It always amazes me how many people / organisations don’t do this properly or even at all.

You know how annoyed you get when your mum tells you she isn’t backing up all her family photos – well that’s what I get like when I see IT systems neglecting backups.

Note this post refers to a standalone Tableau installation with a manual failover to DR. We don’t yet have a clustered environment. I’ll update the post with considerations when we implement that.

 

What’s a backup?

Seems a simple question, and there are a number of different types of backups that you can take, each useful in different situations. Here’s what I’ve got in place:

 

 

Full System Backup

This is a complete dump of the server filesystems to disk (or tape – there’s still plenty of tape backup infra out there). Most likely it will be one of the big vendor products that look like the mothership from Close Encounters of the Third Kind.

Your full system backup should be set up by your server team when you get your machine. However, the principle of “trust no-one” applies here as always and it’s up to you to check the following:

  • Have the backups been set up at all?
  • Are they backing up all the filesystems? – Many times I’ve seen that only operating system partition backups have been set up, and I’ve had to request the application partitions be included.
  • Have the backups been succeeding? – Get your backup team to send you a report every month of backup completion. They don’t always succeed and you probably won’t be told that there has been a failure.
  • If you need to perform a restore, do you know the process and how long does it take?

If you get the okay on that then you’re good. But only as an insurance policy. Full system backups can take a long time to restore, and may only be weekly so you could end up losing data even if these are in place. It’s up to you to ensure you’re covered rather than rely on other teams doing things correctly.

 

 

Nightly Tableau Backup

There’s no excuse for not having this in place. It’s easy to set up and it is a case of when rather than if it saves your ass.

The tabadmin backup command gets Tableau Server to dump all content & configuration to a single .tsbak file. You don’t have to stop the server to do this and it doesn’t seem to impact performance too much while it is running so this should be the first backup you configure.

A simple script like this will do the job.

@echo OFF
set Binpath="D:\Program Files\Tableau\Tableau Server\9.0\bin"
set Backuppath="D:\Program Files\Tableau\Backups\nightly"
echo %date% %time%: *** Housekeeping started ***

tabadmin backup %Backuppath%\ts_backup_ -d
timeout 5

tabadmin cleanup

move "D:\Program Files\Tableau\Backups\nightly\*" \\\tableau_shr\backups\nightly\
echo %date% %time%: *** Housekeeping completed ***

The tabadmin backup command does the actual work here, dumping everything to a file. Always a good idea to run tabadmin cleanup afterwards to remove logs etc.

We run this script at a quiet time for the server (not that there is one in my global environment). We use the Windows Scheduler on the server but I’d recommend using a decent scheduler like Autosys or whatever your enterprise standard is as WTS is pretty poor.

IMPORTANT: You may have noticed the move command at the end there. That takes our newly created backup file and moves it OFF THE SERVER to a share drive accessible by my backup server. Why? Well what happens if you lose the server and your backup file is on it? You may as well have no backup. So move it somewhere else.

Update – this tip actually saved my ass this week when we lost our entire VM cluster (er.. hardware team – *cough* – what’s going on??) . We were able to failover to the backup server successfully. Going forward we will be soon implementing Tableau’s High Availability capability.

Do make sure you rotate your backup files with a script that deletes the old files or your share drive will fill up. I keep 4 days worth, just in case the current file is somehow corrupted – rare but can happen.

 

 

Weekly Restart

You may know I’m not a fan of running enterprise apps on Windows. I prefer Linux for a number of reasons that I’m not going to go into here. I know many users want Tableau Server on Linux, and the amazing Tamas Foldi has only gone and written it himself – so one day we may see it.

Anyway, with Windows apps I always build in a weekly application restart. In our case every Saturday morning. That involves a server reboot (to clean out any OS related temp stuff), application restart and a tabadmin cleanup. The tabadmin cleanup with the server stopped has the added bonus of clearing out the temp files (doesn’t happen when the server is running). These files can get pretty big so worth clearing out.

 

 

Virtual Machine Snapshots

If you’re running on a VM then you may be able to utilise the VM snapshot facility. Contact your VM admins for details. I’ve not needed to implement this but I know some that do. VM snapshots are super handy.

Do be aware that Tableau don’t seem to support this though..

Screen Shot 2015-11-11 at 19.11.48

 

 

Config File Backup

Sometimes it’s handy to just back up your Tableau Server config. I’ve got a script that grabs all the .yml and template files in my Server directory, zips them up and moves them off the server. Pretty useful to refer back to old config settings if you need to. Make sure you include workgroup.yml.

If you’re being really good then you’ll be checking your config files into a revision control repo like SVN.

 

Site Specific Backups

Tableau Server allows you to backup per site. This doesn’t give me much extra but I know in orgs that have lots of sites, or a site per team / business unit it can be very handy.

One thing that isn’t great about exporting a site is that the site is locked and inaccessible as the export is taking place. See Toby Erkson’s blog for more info on exporting a site.

 

 

Backup File Size & Duration

As your environment grows you’ll need to be mindful of the size of your backup file. Mine is around 16GB and takes well over an hour to write. Takes about 25 mins to restore. You’ll need to understand those numbers as your system matures.

unnamed

Backup files can get pretty big

Another variable that can affect backup time is the specification of your primary server. If your primary is low spec then you’re gonna get a longer time to write a backup. I don’t have any stats on that but I know it is true. Contact Jeff Mills of Tableau if you want more info on Weak Primaries & backup times.

 

 

Backup Your Logs

Less important this, but handy to do on a weekly basis is to zip up your logs. We have a much better solution for logfile management using Splunk – you’ll see a blog about that in the future.

 

 

The Most Important Bit – TEST YOUR BACKUPS

OK so you’re backing up like a man / woman possessed? Fine. You’re only as good as your last restore. So TEST your backups periodically. Files get corrupted and you don’t want to be discovering that your only backup is broken when you need it.

OK that’s it. Backups can save your life – don’t ignore them. Paranoia is king in IT!

Cheers, Paul

How to manage Tableau upgrades in an Enterprise environment

Hi,

Been a while since my last post. Been exceptionally busy at work due to widespread adoption of Tableau at my organisation. Usage has doubled in the last 3 months and we now have thousands of users to keep happy. That takes some doing, hence the blog hiatus.

Anyway, time to continue the series on Tableau as an IT Service, with a subject that I’m asked about a lot – just how do you manage your Tableau upgrades in an Enterprise environment?

This is actually a pretty big subject, especially in an Enterprise setup. There’s no perfect way to do it but hopefully some of these tips will be useful.

 

Section 1 – Pre-Upgrade Considerations

To upgrade or not? That is the question

Obviously you’ll need to make some sort of decision as to whether or not you actually need to upgrade. Each time you upgrade a production system you risk impacting stability, introducing bugs or human error. It also needs testing, planning and eats resources and time. So if there’s no good reason to upgrade, then don’t.

Tableau release new versions at a pretty impressive cadence, generally once a month for ‘maintenance’ releases. So for each newly advertised release, take note of the following in order to make your decision.

  • Compelling functionality –  New features are entering the product all the time. Determine what may be useful to your user base.
  • Key bug fixes – Each new version will squash a few bugs. If there’s one that is affecting your users then it may be prudent to upgrade and quieten the noise. Remember that your upgrade may introduce new issues. Take note of the known issues section in the release notes.

Both of these are fully documented in the Release Notes. Review them carefully each time Tableau announce a new version. There are occasions where bug fixes are not announced in the release notes but your account manager will make you aware of those.

Also be aware that new versions might also introduce new bugs / issues. We have had situations where we have been stable, decided to upgrade and then spent the next few months battling newly introduced issues to the point that we probably should have stayed on the older version.

Be aware of compatibility issues

I hear a lot of complaints about compatibility issues between Server and Desktop. So it’s important to be aware of the behaviour between versions. Get this wrong and you may be in a position where users have overwritten content originally created in an older version and don’t have a back up to roll back to. If you are crossing major versions (8 -> 9) for example, then you’ll certainly need to upgrade the Server and Desktops at the same time.

Top tip – it is possible to hack the xml of a workbook to change the version and rescue the situation, provided no edits have been made to the content. The ever-so-talented Jen Vaughan (@butterflystoryexplains all here.

For more details see this article.

Don’t upgrade to version zero

Most risk averse IT managers (like me) will resist the temptation to jump right into that new shiny Vx.0 release the day it comes out. Version zero releases of any software are notorious for bugs and issues, that’s just the nature of software development. So at my org we always let at least one maintenance release slide by and instead go to the Vx.0.1 or Vx.0.2 release.

It can be hard to resist temptation, especially when your users are clamouring for that shiny new version but if there are major bugs in that zero release, then best let someone else find out about them rather than you.

Caveat: This doesn’t always work of course. We waited for 9.0.2 and that ended up being one of the buggier releases. Oh the irony.

 

Understand the new version resource demands

This is important. You may be rocking away on your existing version, confident that your hardware can satisfy the software. But then you upgrade, and all of a sudden that new version eats up double the RAM or batters your CPU. You didn’t see that coming.

For version 9 Tableau released an updated scalability document. Annoyingly it was released quite a while after V9 went live. I was expecting a comfortable read but noticed phrases like the following, which were pretty alarming.

Screen Shot 2015-09-04 at 23.03.31

Screen Shot 2015-09-04 at 23.05.20

Whaaaat?! That led to some discussions with Tableau tech folks (thanks Meredith!) and some fevered testing and all turned out to be ok. But those figures took us by surprise for sure. Don’t be in a position where you upgrade and then suddenly hit a capacity issue that your older version didn’t have.

 

Test it!

Let’s say you upgrade. And it goes wrong. One of the first questions you’ll get asked is “Did you test this?”. You really don’t want to be answering “no” to that question.

You should have at least one non-production environment that you can run tests on. Due to the complex nature of Tableau it is impossible to test every aspect of functionality but you should at least be able to cover a good number of scenarios. You may also not be able to simulate your production load on your test environment but it will be better than nothing. You may find the load testing utility tabjolt handy here. Check Russell Christopher’s guide to tabjolt.

I have the following environments to play with. This gives me a lot of options.

  • Production – Main user facing environment
  • BCP – Disaster recovery environment in case production fails
  • UAT – A mirror of production. Same spec
  • Engineering – Lower spec, used to test the latest version available from the vendor
  • Beta – Even lower spec, used for testing beta versions
  • Alpha – for testing the alpha versions

Ensure that the tests you conduct are consistent, repeatable and that the outcomes are recorded. We use a tool called Quality Centre and the tests are performed by my level 2 support folks. This gives consistency and frees time for my main analysts.

Verify your licence details

Double check your licence maintenance end date for both Server and Desktop. If you’re out of maintenance then you won’t be able to use the application after you upgrade. I’ve seen licence issues way too many times with many applications after an upgrade. You won’t want to be trying to contact your account manager on a weekend to sort out a licence issue.

Opinion – IMHO I would much prefer it if applications didn’t crap out due to licence expiration. In 99% of cases there’s some paperwork misunderstanding that is easily sorted by your account manager. By all means let the application hit you with some warnings and also alert the vendor, but it shouldn’t mean a loss of service.

 

Your upgrade process & strategy

I’m not going to go into it here as it’s a full on subject in itself but make sure you follow your organisation’s Change Management procedure to the letter. Failure to follow change processes is generally a dismissible offence in most Enterprises.

Make sure you advertise your strategy for upgrades and maintenance to your users. You’ll get asked, so ensure this is specified in your Service Document. You may even have standard maintenance windows (e.g. on a weekend) where you can reserve the right to take down the system. Again, make sure that is documented and your users are aware.

 

Section 2 – The Upgrade

Create a package

Most enterprises will use some form of package deployment tool / team to perform the actual deployment of new software. That’s pretty handy. I have over 500 installations of desktop to support and we need to ensure they all get upgraded at the same time. So I can create a software bundle, send to the packaging team and they will then schedule and deploy.

This gives you the chance to include those little extras in your package to give your users the best experience. Here’s what’s in our Tableau Desktop package.

  • The installer exe file
  • A sample “Getting Started” workbook with tips, best practices & help.
  • Preferences.tps file containing customised colour pallettes
  • Most used drivers (Oracle, MySQL etc)

I would love to be able to customise the “Discover” pane to point to some of my internal resources but it doesn’t seem to be possible. Boo.

We also supply custom instructions for the packaging team, such as running the installer with a flag / registry update to disable the auto-update feature that has been implemented in the upcoming 9.1 release. A really baaaaad idea for Enterprise deployments.

One thing to be aware of with packaging is that it can take a long time. From request to deployment, the typical time at my org is an insane 7 weeks! By which time another version is out. We are hoping to speed that up a bit obviously.

Communicate to the max

You can’t over communicate any potential disruption to your service. Make sure a broadcast message goes to your users via whatever system your firm uses. And it doesn’t hurt to follow-up with your power users / senior stakeholders with a personal reminder that work is planned.

Take a backup

Whadda ya mean you didn't take a backup?

Whadda ya mean you didn’t take a backup?

Tableau is one of the easiest applications to upgrade that I’ve worked with. A simple uninstall / reinstall does the trick. But don’t take that for granted – make sure you take a manual backup prior to your upgrade. If you’re not doing this as a matter of principle then you’ll be getting a visit from the boys. And they won’t have had their dinner.

Handily, the uninstall process takes a backup anyway but don’t rely on that, take your own.

You should also back up all your .yml configuration files and ensure you know what each setting is. Tableau should preserve these settings during the upgrade but just in case it doesn’t then it’s handy to have a copy to refer back to.

Server specific considerations

When you uninstall Tableau Server it backs up the content and the settings in the main yml configuration files. That’s cool, but do remember that if you’ve changed any of the other config files then they will be overwritten and you’ll have to make the changes again. For example we change the webserver timeout settings in the file “\Tableau\Tableau Server\data\tabsvc\config\httpd\httpd.conf.templ” – that gets blown away by an uninstall.

There may also be other settings in the Postgres db that you may have modified using tabadmin. Not all are retained from what I can see. Note I’m still researching this so not 100% sure of the behaviour.

Finally make sure you understand any changes to the Tableau Postgres DB schema in the new versions. It has generally remained pretty consistent but if any tables or fields are renamed then that may well break your Custom Admin Views.

Section 3 – Post upgrade

Test it! Again!

Not all issues come to light immediately. Perform testing, keep vigilant and liaise with your users closely in the next few days to understand if the application is behaving as it should be.

Ask for help!

Tableau Upgrade Assistance

Tableau Upgrade Assistance

If all this sounds a bit daunting and you’d like to get assistance then Tableau offer an “upgrade assistance” programme which might be worth looking at. Talk to your account manager for more information.

There are also other guides around. Have a look at this one from our good friends at Interworks.

That’s it for this post. As I said it’s a big subject so do post comments if you feel I’ve missed anything.

Happy upgrading! Cheers, Paul

How to Monitor Your Tableau Server – Part 2 – Tableau Server Application Monitoring

Hello there,

Following on from Part 1 of this series. Here’s part 2, how to monitor your Tableau Server application itself.

Now I don’t know server in as much detail as some of the Jedi-level experts out there so I’m totally open to different ways of doing things. My recommendations here are based as much on general IT service monitoring best practice as they are on Tableau specifics. If I’ve missed something then do point it out – hoping the community can help me expand this article. 

On that subject – I’m delighted to have been able to collaborate with Craig Bloodworth (@craigbloodworth), Mark Jackson (@ugamarkj) & Chris Schultz (@nalyticsatwork) on this. Thanks for your invaluable contributions guys.

Are we ok? That’s the ubiquitous question on an IT service manager’s mind. And it can be a real worry. But the fact is that there are a lot of tools and methods you can employ to cut down that worry and stress or even eliminate it.

 

Service Availability

Simply put, is your Tableau Server up or down? Tableau offer a “Server Status” view, but in my opinion that’s pretty useless as you’re never going to be staring at it for the whole day. I’m also not sure how quickly it updates or responds to the system activity. It never seems to change when I’m looking at it.

status

Tableau’s Default Server ‘Monitor’

So it’s clear you’ll need something else to give you that early warning of any issues.

xml

Tableau Server Monitor in xml

Btw you can also get this in xml output. Could be handy.

 

Process Monitoring (Enterprise Process)

procs

Main Tableau Server Processes (click to enlarge)

These are the key processes (running programs) that are required for Tableau Server to function. If one of these has crashed they you’ll likely have a problem.

So referring back to Part 1, I talked about enterprise monitoring tools used to monitor your Tableau infrastructure. Well you should be able to use these tools to set up application monitoring. That’s monitoring of your own application, that you define (and ideally configure) that produces alerts that come to you or your own support team (via the enterprise process).

You should set up monitoring rules to alert on zero instances of each of these processes. The alerts need to be classed as a “Critical” severity so that they hit the alert list of the Level 1 team (non-critical alerts may not be visible). Make sure the monitoring rules apply 24 x 7.

Important – Make sure that the Level 1 & 2 teams that will get these alerts know exactly what to do with them. These teams will probably have a document or Runbook that you’ll need to fill out which will give them instructions as to what the alert means and who they should call. This needs to be crystal clear as they’ll usually follow it to the letter.

Process Monitoring (Paranoid Android Process)

marvin_660

“I knew that alert would get lost. Don’t say I didn’t warn you..”

So even if you set up the above monitoring using the Enterprise Process, then you may have issues. That process can break, meaning that your alert may take up to 30 mins to get to you (or a lot longer!).

Therefore I always encourage being as paranoid as possible when it comes to monitoring.

Luckily there are a number of things you can do to add an extra level to your monitoring.

 

Use a Simple Script

miker

Monitor Tableau Server without the GUI

Mike Roberts of Interworks has written a simple guide to scripting up a basic process check based on the default Tableau Server monitor xml output mentioned above. You can run that script using Windows Task Scheduler and get an email if any of the processes are detected to be down.

 

 

I don’t use that one, but I do have a very basic Powershell script that I run using Task Scheduler every 5 mins. Does the same thing. It’s based on the following code.

powershell.exe -command "& {if (! (get-process -name postgres -erroraction SilentlyContinue)){Send-MailMessage -SmtpServer '' -from  -To  -Subject 'postgres.exe not running on PROD '}}"

All that does is execute in the background and if the process name (in this example postgres) is not detected by the get-process command then it sends an email to my team. Not foolproof but when combined with the enterprise process then it gives me a better level of protection.

Query the processes via URL

craig

Querying processes via URL

This is a new one on me. Apparently it is possible to query each process by http and get a message back to indicate if the process is ok. Opens up a lot of options for more scripting of remote checks or monitoring of the URLs via third party applications. All adds to the arsenal of monitoring available to the service owner. Many thanks to Craig Bloodworth (@craigbloodworth) of The Information Lab for this tip. You can find more details in this blog post.

The Windows Event Log

By default Windows will log any messages or errors to the Windows Event Log. This can include system and application alerts and is a great source of data regarding system health.

tableau_event_restartingdeadcomponent

Windows Event Log (click to enlarge)

Fire up the Event Viewer (somewhere in administrative tools menu usually) . You should see a number of categories of event on the left, from system stuff to specific application messages. Some will be informational, others downright confusing, but there will be some gold dust in there that you need to be mindful of.

For example – the image (right) shows that Tableau server has been restarting the backgrounder process due to a crash. That’s not critical to know about immediately but I’d sure be interested to understand if it is happening regularly.

There are ways you can export this data automatically and then create a Tableau datasource – we haven’t done that yet but are planning to.

Windows Performance Monitor

perfmon

Windows performance monitor data collector

You can also make use of the inbuilt Windows performance monitor to collect and export data regarding the performance of the Tableau processes on your server. We set up a collector and constructed a basic Server Health Dashboard.

 

 

 

server health

Server Health Dashboard based on Windows perf mon stats

It’s a good idea to subscribe to these dashboards to get them dropped in your inbox at the start and end of your production day.

The details for setting this up are on this Tableau KB article.

 

 

Tableau Log File Monitoring

To me the Tableau logs seem like a real mystery. There’s clearly a ton of information in there, but even the Tableau support folk don’t seem to know what’s important and what’s junk. There are also a lot of messages that seem like red-herrings and some that are just plain confusing.

It’s a shame that there isn’t more clarity on which strings and messages we should pay attention to, at the moment I’m just guessing.

In terms of alerting, the enterprise monitoring tool you use will have an equivalent log scraping functionality, just as it does for process monitoring. This will involve you telling the tool which text to alert on. Fairly simple. You can also write your own script in much the same way as the powershell process monitoring script mentioned earlier in this post.

I get really annoyed with the state of the Tableau Server logs. They’re a total mess. There are multiple locations, and there’s little consistency. I’ve not had time to analyse them properly but it seems like some entries contain either DEBUG / INFO / ERROR or FATAL which would give an indication of whether you should trigger an alert based on the occurrence. It doesn’t seem consistent though.

Ideally I’d like every log entry from every component to start with a timestamp, then either of these severity indicators. Would make it so easy.

 

Log analysis using Splunk

splunkIf you’ve not seen Splunk then you should take a look. It’s a great tool for aggregating and analysing masses of log file data and is in widespread use at many large enterprises. I don’t use it yet but it’s in the pipeline.

Another bit of collaboration – Chris Schultz has written a guide to using Splunk to analyse Tableau Server logs. It’s on his new blog here.

 

Monitoring Tableau Server Activity

Monthly Server Stats

A wealth of info is available from the Postgres DB

So you’ll probably know that Tableau has an internal Postgres database. You may not know that you can interrogate this database easily and pull out pure gold! It’s an absolute treasure trove of information about your server performance, usage and pretty much anything else.

I’m not going to elaborate on it here as my good friend Mark Jackson (@ugamarkj) has written a comprehensive guide on it here.

This is critical ammo to the Tableau Service manager and making these dashboards available to your user community will get you some serious brownie points, especially with senior management. Most applications don’t have the ability to provide this level of detail, Tableau does, and it’s a great feature.

Other Resources

As mentioned there are a ton of ways to do this and there are many more guides out there. Take a look at some of these links.

http://www.alansmitheepresents.org/2014/02/tableau-server-performance-monitoring.html
http://kb.tableausoftware.com/articles/knowledgebase/automation-checking-server-status
 

OK that’s it for this part. Hopefully that’s given you an idea of what is possible in terms of monitoring the Tableau Server application. Got any ideas or methods of your own, then do share!

Cheers, Paul

How to Monitor Your Tableau Server – Part 1 – Infrastructure Monitoring

Hello there,

I hope you are all well and recovered from #data14. What a great event that was.

I’m gonna get a bit serious on yo now. It’s time to talk monitoring.

For a Tableau service manager (or any IT service for that matter), the worst situation that can possibly occur is getting a phone call from your users to tell you that your service is down. At best you’ll look stupid, at worst it will cost you credibility and is a sure-fire way to destroy user confidence in your service.

So how do you avoid this? You could not have any outages – well you can forget that, it aint gonna happen. You’ll get issues so get ready for them. What you can do is monitor your service big time. That way you’ll get the heads up and you can answer that phone call with a “yep we know, we’ve just raised an incident ticket and we are on it” – or better still, get to the incident and fix it before users even notice! Remember that effective incident management can actually gain you plus points from your user base, and senior management.

The problem with monitoring is that it’s BORING. I should know I did it for 12 years! But it’s also essential! Get it right and you’ll be making your life a lot easier. It also traditionally doesn’t get a whole lot of investment thrown its way as there’s no immediate tangible business benefit.

Monitoring falls into these categories. This is likely to take me more than one post to explain and it’s a big subject so I’ll doubtless miss some bits out. As always, I’m happy to connect offline and explain.

  • Infrastructure monitoring
  • Application monitoring
  • Performance monitoring
  • Capacity monitoring
  • User experience monitoring

Infrastructure Monitoring

As the name suggests this is all about monitoring of your infrastructure. That’s your hardware and network, peripherals and components of the platform your Tableau Server application is running on.

Chances are the infrastructure will be owned by an IT team. You’ll need a great relationship with these folks so if you haven’t then start buying them some doughnuts now. From what I can see Tableau is often brought into organisations by business users and that then antagonizes IT, meaning this relationship isn’t always the best. That’s a separate conversation however.

 

How does infrastructure monitoring work?

Chances are your monitoring team will have decided on an enterprise monitoring tool for the whole organisation. It will probably take the form of a central server, receiving alerts from an agent that is deployed as standard on each server in the estate.

NagiosSome examples of commonly used monitoring tools include the following. I’ve got a fondness for ITRS Geneos myself but am not going to go into the relative merits of each tool. You won’t have a choice what tool is used in your org anyway.

So what happens? Well the agent will have a set of monitoring “rules” that it adheres to. These will take the form of something like “check the disk space on partition X, every Y minutes and trigger an alert if greater than Z percentage full”. That’s all the agent does. Polls the server for process availability, disk space, memory usage etc on a scheduled frequency and triggers an alert to the central server if the condition is breached. Those parameters should be fully configurable.

consoleThe central server will then display the alert on an event console such as this one (pictured). Alerts will be given a criticality such as minor, major or critical. The alert console will be viewed by a support team, usually an offshore Level 1 team that provides an initial triage of the alert. They may then pass it onto a Level 2 team for potential remediation, or they may also pass it on to Level 3 – the main support team. That’s the usual process in a big organisation.

So what’s the issue with that? Well there’s the time factor for one. It can sometimes take 20 – 30 mins for an alert to get to the person that matters. That’s obviously not great. Also there’s the sheer volume of alerts, a big organisation can be dealing with tens of thousands of active alerts a week, many of them junk. That increases the risk of your alert being missed. There are also a lot of break points in the process, and sometimes alerts just go missing due to lost packets, network issues etc. It happens. On the whole the process works though.

 

Who’s responsible and what for?

Your infra teams are 100% responsible for the monitoring of these components. This encompasses

  • Server availability (ICMP ping)
  • CPU usage
  • Memory usage
  • Disk space (operating system partitions only)
  • Network throughput / availability

trustnooneThey’ll tell you not to worry about this. They’ll tell you that any alerts will go to their support teams and they’ll be on it should they detect an issue. My advice – don’t trust anyone. There have been many times where I’ve had an issue and lo and behold the monitoring hasn’t been configured properly, or hasn’t even been set up at all. Or there’s been a bad break in the process somewhere. That aint cool.

 

So what should I do?

Take these steps to keep your infra teams on their toes. They’re providing you a platform, you are entitled to ask. They might not like it, but stick to your guns – you’ll be the one who gets it in the neck if your Tableau Server goes down.

  • Ask for a breakdown of the infra monitoring thresholds – What’s the polling cycle for alerts? What thresholds are being monitored? Who decided them and why?
  • Ask for a process flow – What happens when an alert is generated? Where does it go? How long does it take for someone to get on it? How is root cause followed up?
  • Ask to have visibility of the infra changes – If there are changes going on to the environment that might affect your server, make sure you get notified. Make sure you attend the appropriate change management meetings so you know what’s going on.
  • Ask for a regular report on server performance – There will probably be a tool on the server that logs time series data on server performance. That should be accessible to you as well as them. Chuck the data into Tableau and make it available to your users.
  • Understand the infra team SLA – It’s important to realise that you are a customer of the infra teams. Ask them for a Service Catalogue document for the service that they are providing. Understand the SLA that they’re operating to. Don’t be out-of-order, but if you find they’re not giving you good service then don’t be scared to wave the SLA.
  • Ask for a report of successful backups – Just as important as monitoring
  • Ask for the ICMP ping stats – How many packets get lost in communications with your Tableau server? How many times does it drop off the network?
  • Be nice – The infra teams in big orgs have a tough job. They’ll have no money and little resource. Cut them some slack and don’t be a prat if they let you down occasionally. It happens.

Start with that lot. Your users will also love it if you can make this information available to them. Again, it inspires confidence that you know what you’re doing.

OK that’s it for infrastructure monitoring. Next up I’ll dive into how you monitor your Tableau Server application.

Cheers, Paul

 

Tableau as an IT Service – Part 4 – Operational Considerations

Hi all and welcome to Part 4 of the seemingly never-ending story called Tableau as an IT Service. Lot of good chat and engagement on this from the community, thanks in particular to Paul Chapman (@cheeky_chappie), Matt Francis (@matt_francis), Kelly Martin (@vizcandykelly), Robin Kennedy (@odbin) and Francois Ajenstat (@ajenstat) for their interest and support.

Must admit I didn’t realize I’d be writing this much on the subject when I first thought of the idea but a number of people have asked for my opinions on a few extra areas so I’m trying to cover them wherever possible. It’s certainly a big subject and one that if done incorrectly can lead to even great tools like Tableau being seen negatively by your user community. So it’s important to get it right.

So far in this series I’ve covered the following aspects

  • Project Initiation
  • Infrastructure Setup
  • Tableau Server Configuration

Those are all well and good but a robust IT service will also have its operational ducks in a row, and that’s the subject of…

Tableau as an IT Service – Part 4 – Operational Considerations

What I mean by “operational” is how your service is performing when you’ve given the green light to your users and people are now actively using it. There are a number of subjects to consider in this space, some of which can make or break an IT service, irrespective of the application involved.

Application Registration

I didn’t really know what to call this category. What I mean is that every new service will need to be registered in a number of other IT systems for it to be fully integrated into your organisation. For example.

  • Payment & purchasing systems – So that accounts and procurement teams can find the vendor details in their directory and ensure payments get through to the right department.
  • Application directories – So that you can accurately track change to your environment and your application is correctly reported on to the business. It will probably be given an “application code” or something cryptic.
  • Business Continuity systems – Required by BCP and control teams to ensure you tick the box for having appropriate control and disaster recovery functions and that your systems are involved in any testing events.
  • Information Security systems – Similar to the above but with regards to InfoSec checks and records.
  • Architecture & application strategy – Make sure that the appropriate architects and strategy departments are bought into Tableau as a strategic tool for the organisation. If they don’t know about it you might find them recommending another tool instead and then you’ve got a fight on.
  • Appropriate contact lists – Make sure your service is listed in any emergency procedures and contact directories. If something fails and users are impacted then they need to know who to call.

This all ensures that when you enter “Tableau” to one or more systems to perform task X, then you get the appropriate details rather than “Unknown application”. Some of these systems may be global or have regional variations.

Chargeback

This subject has provoked a number of questions to my Twitter account. In truth we began our Tableau service without a chargeback model. This decision was primarily due to the small-scale of our initial offering in terms of user licences and also due to the desire to make user adoption as hurdle-free as possible.

Now that our user count is growing and the server usage also increasing, we are looking to implement a recharge model. Chargeback is commonly used by IT departments as a way of charging business units for their utilization of IT systems.

There are a number of advantages for IT departments.

  • Cost visibility – With a solid chargeback model a service owner can easily report the usage of a system or service back to senior management, with an associated cost value. This can be used to better understand the overall cost picture of IT in an organisation.
  • User control – If users have to pay for their level of usage of a system then they are much more likely to use the system responsibly and only use what they actually need. That way service resources are more efficiently utilised. This can also be problematic if the chargeback model is flawed (see disadvantages below).
  • Service perception – A service with a clear chargeback model is more likely to be adopted as a longer term IT strategy by senior management. Being able to demonstrate financial transparency and clear understanding of how much a service costs to run, and how that cost will change with service expansion is manna for senior management and budget controllers.

And some disadvantages

  • Reporting & admin – Every model will force users or service owners to regularly and accurately report their usage to management. This can take a while and can rapidly become onerous.
  • Abuse – Chargeback undoubtedly forces users to change their behaviour. But get the model wrong and you can suffer. I’ve seen cases where job scheduling tools are charged per job, rather than per server or user and that has led to user teams consolidating huge batches of jobs down to a small number of commands, executed by one job group that forks off hundreds of other commands. Saves money, yes – but extremely bad IT practice and ultimately bad for the system health. But if their goal is to save money you can understand how that can happen.
  • Shared components – How to charge back on components that are shared between multiple teams or applications? Can get tricky. Usually a balance is implemented to reduce admin overhead but this can mean one team paying more or less than they should.
  • Adoption – Users are more likely to give your service a try if it is free. Simple. And if your chargeback model is flawed then that could mean your service doesn’t fly at all.
  • Power users – With any service you’ll get your power users. The ones that push you for the upgrades and test the functionality to the limit. If they’re making you spend more on the service for functionality that other teams don’t require then there’s sometimes a case for applying a tiered model to charge them more. You want it? Then you have to pay for it. It’s usually a good idea to keep power users happy as they’ll sell your service for you and help to realise the business benefit.

As stated, we don’t actually have a chargeback model for Tableau yet. It’s currently under discussion.  My initial thoughts are to charge according to either disk space used on the system or by user accounts. I’d be interested to hear how the community is doing this. Let me know what’s been working (or not) for you.

For other services that we look after we tend to charge back on the number of user accounts on the system associated with a particular business unit. Or a flat charge per server using the service. This was done with ease of admin in mind and I think it could do with a review as some teams are paying more than they should and others are getting a bit of a free ride.

Onboarding

So how do you get smoothly and seamlessly from “Hi Paul – I really want some of that Tableau action” to “Cheers Paul – I love it, thank you for bringing Tableau into my life”?

Here’s what you need to consider in your onboarding process.

  • Speed – It needs to be fast fast fast. I work in an area characterized by agility and short time to market for pretty much everything and my users don’t react well to delays. Anything over a couple of weeks and you’ll risk frustrating your users.
  • Clarity – You’ll need proper documentation and ideally a flow chart of the process. Any costs, especially recurring maintenance costs need to be fully understood. If your systems can provide a link or report for the user of where the request process is at any one time then even better.
  • Ease – Ideally it will be one request via ticket or email to the initiating team. Users hate having to revisit a process and often forget to do so, meaning that the whole request process gets held up. There’s nothing worse than users having to fill out some obscure or unnecessary web form with loads of mandatory fields.

Our process goes like this

  • Initial request – Our users start with an initial email to a helpdesk team who then raise a ticket to track the request. Our documentation states that the email must include approval for the desktop licence cost from their manager. This can be a problem in most onboarding processes as chasing a manager to approve a spend can be a real pain.That’s why we leave that up to the user so that the helpdesk team don’t waste their time tracking down the approval.
  • Ordering – Once the helpdesk get the approved request then they place an order in our ordering system. We use a tool provided by Ariba. That goes off to Tableau and payment gets made using the transit or billing code provided in the request, and the helpdesk get a licence key which is passed on to the user.
  • Install – Helpdesk then send a request to the desktop installation team, who deploy our packaged version of Tableau Desktop to the user workstation.
  • Server – If the user requires a server licence then the helpdesk pass the ticket onto my team, and we add their account to server.

That whole process usually takes about a week, which is generally fine for all but the most demanding of users. Crucially it is only one email that the user has to submit to helpdesk so it’s admin-lite from the client perspective. They love that.

So why is onboarding so important? One of our other enterprise services hasn’t been set up so well and has a particularly poor onboarding process which can take several months. This means that by the time the user gets access to the system they are already pissed off with the whole thing and this means they have a very low tolerance for any issues or problems with the service. Because we get users onto Tableau quickly and smoothly they are happy and keen to get stuck in from the start – and if they do get any issues it means they’ll cut you some slack rather than escalating to your boss.

Provisioning

Most enterprises will have a central mechanism for deploying software. Typically there will be one for the UNIX server environment (Opsware), one for the Windows server environment (SCCM) and one for the Desktop environment, although SCCM covers both servers and desktop in our case. There are a few steps involved in getting a version of Tableau Desktop (or other application) to the point where the deployment team can hit the button.

  • Packaging – Firstly I’ll download the new version from the Tableau website and then send it to the Packaging team. They’ll form it into an msi package and place it on a repository for me to access for testing.
  • Testing – I’ll then have to test the package. Typically I’ll do this on my own desktop or one of our shared machines. Always a good idea to document this process as you’ll be officially signing off that it works so you may need to evidence that.
  • Staging – Once the testing has been signed off as successful the packaging team will stage the package on one of their “deployment points”. That could be anywhere, the location isn’t important although if your organisation has issues with technology consistency (see part 2 of this series) then you might find a platform team in one region doesn’t have access to deployment points in another region.
  • Deployment – The deployment team should be able to use their admin console to easily and automatically deploy the software to the machine(s) of your choice. The requester will get a report back as to the success / failure of each attempt.

Unfortunately even with a packaging process Tableau is a little harder than most applications to distribute due to the frequency of updates. See the “Upgrades” section below for more detail.

Upgrades

Tableau actually make this quite difficult for an enterprise scale environment. As you know they have an active release cycle of approx 1 month per minor release. Now that’s ace for the average user, who can go and grab the latest desktop whenever they want. But not so good for the service owner. As stated earlier we like to package up any application and make it available through the official enterprise distribution channels so that admin teams can control distribution and upgrades. Unfortunately such an aggressive release cycle means that we just don’t have the time to package each release and make it available for deployment as the process can take weeks. So we tend to re-package our desktop version once a quarter, or if the update has a critical bug fix etc. It’s probably ok if your organisation has an agile packaging method / team but in my place we just didn’t have the resource to keep up with Tableau’s release cycle.

Unfortunately there is also nothing stopping users from upgrading their own desktop application without the knowledge of the service owner. I’ve been contacted by users asking why they are suddenly unable to publish to server, and the answer is because they have upgraded their desktop to server version +1. Luckily Tableau desktop installations can co-exist happily so long as you’re aware what version you’re saving a workbook with. The same challenges exist with Tableau Server upgrades, although these can be even more time-consuming to implement.

All this means we tend to hold back on upgrades unless there’s an important bug fix or significant functionality jump.

Support of the Service

Obviously your service will be supported. But that support model can vary in a number of ways, not least dependent on the amount of resources that your organisation is able to throw at it. The chances are that whatever team supports your Tableau service will also have to support other applications as well. There are a number of key considerations in this area.

  • Service Tier – First off, what tier of service are you offering? If it’s a lower tier say 3, then support will most likely be during business hours only, with no on-call service to fix issues in the middle of the night or weekends. Anything higher and you’ll have that in place. If the business is demanding the service be a higher tier then they’ll have to provide you the resources to deliver that service. There are occasions where that doesn’t happen and it’s a recipe for disaster.
  • Team setup – In your support team you’ll obviously need one Tableau subject matter expert (SME). If you have more people then one of the first things to do is make sure that knowledge gets transferred. The others don’t need to be experts but they do need to be able to address issues and keep the system going when the SME is absent. This all sounds obvious but I’ve seen many a team that is left floundering with a huge skills gap when someone leaves or is off sick.
  • Documentation – There’s no excuse for not having this in place. Key admin tasks and  a list of the common issues and remediation steps should be easily accessible on a sharepoint or confluence page. You’ll be amazed how even an expert’s mind can go blank when faced with an incident. Your docs should also be detail the environment and the appropriate governance. If you’ve done the right things in terms of project initiation (part 1 of this series) then you’ll be ok here.
  • Alert Processing – When your monitoring system generates an alert how does that alert reach a support person? It could ping an email to a support Blackberry carried by the on-call technician, or it could go to a central Network Operations Center (NOC) who would view the alert on a console such as Netcool Omnibus or HP Openview. They would then call out the appropriate technical resource or conduct some initial remediation actions, thus potentially avoiding the need to wake you up. If you are using a NOC then you’d better ensure your alerts are correctly raised, free of junk alerts,  as that team will be calling you up if they get one, regardless of how minor it may seem.
  • Support Flow – Just like the onboarding flow, you’ll get some kudos from your users if you can give them a flow chart that documents exactly what happens from the moment an issue is detected to the resolution.
  • Budget – Make sure you’re aware of any ongoing costs to your service and think well ahead. If you think you’re going to need some more resources either in terms of tech or people then GET IT IN YOUR BUDGET PLAN as soon as you can. Most budget cycles operate over a year in advance and managers hate any unplanned costs appearing when all the numbers have been signed off. Even if you might not need the spend then it’s best to plan for it as it can always be removed.

Luck plays a part in the success of your service support. Some applications are more prone to issues than others, some take more maintenance and some are just plain flaky. Luckily Tableau doesn’t seem to suffer from any of those problems, giving your team a chance to concentrate on helping the users make the best of their experience.

Server Performance

Tableau make a number of recommendations in their documentation on how to improve performance if you have an issue. That’s fine, but you want to be addressing performance before you have an issue. That’s the job of your application monitoring tool (more details in part 2 of this series).

In my experience I’ve seen poor dashboard design and inefficient queries to be more of a problem than actual issues on the server side.

While you’ll get an idea of CPU and RAM usage trends I don’t see any way that Tableau can proactively alert if extracts are taking longer than expected or if queries are running slower than usual. Ping me if you know of a way. I’d like to be able to specify the expected time of a query or extract refresh or ideally have the system work that out based on historical information and alert me if a query or extract has breached the usual threshold.

Capacity Management

This is another tricky subject and one that we haven’t mastered yet in my service. From what I can see there’s little to stop users uploading massive extracts to your system and filling up the disk unless you have a checkpoint in your service model where publication of user workbooks has to go through a central team for approval and analysis in terms of best practices. My team doesn’t have the resource for that kind of setup, you may be more fortunate.

Any monitoring system deployed on the Tableau server (see part 2) would be able to detect when the disk space thresholds are breached on a machine. That may give support teams enough time to prevent the disk filling up.

Interested to hear the opinion of the community on how best to manage resource capacity for Tableau server. I think the best way is probably to make it a procedural checkpoint as described above. That obviously takes resources.

In terms of the infrastructure, your platform and storage teams should have capacity management tools and strategies for ensuring the availability of their infrastructure as the environment grows. It’s not easy to proactively keep on top of capacity as there are so many factors involved, but if your infrastructure teams have solid Key Performance Indicators (KPIs) then they may even be able to use Tableau to visualise the trends and head off any issues well before they happen. That’s what we do at my place.

Incident Management

So what happens when it all goes wrong? You’re gonna get incidents and outages so expect it. And the chances are your users will also understand that systems do break occasionally. What the service owner needs to demonstrate is that they have a good grasp of the impact of the issue and that the resolution steps are clear and pursued as efficiently as possible. You’ll also need to make sure that any follow on actions are performed to avoid a repeat of the same issue. Repeat problems don’t go down well.

Most enterprises will have a dedicated team to handle incident management although depending on the severity of the problem they may not be involved in your issue. So it’s likely there may be a chance that you are managing the incident. Here are some things you’ll need to consider.

  • Logging – Make sure the first thing you do is log a ticket in your incident management system and ensure your manager is aware. This ticket should clearly indicate the issue, impact, steps taken so far.The ticket system *should* send a message to your users but you may need to ping them a note anyway.
  • Incident manager – Ensure someone is in charge of the incident and managing it.
  • Fix the issue – Get the appropriate technical teams on a call and get to the root of the problem. The first priority is to get the system up and running again. If you need to log a ticket with the vendor then do that immediately, sometimes it can take an hour or so for them to get back to you depending on your level of support cover.
  • Close out – Once you’re back in the game get the communications out to your users and management.
  • Follow on – If it’s a serious incident then you may have a “Post Incident Review” with key managers in the following days. This will typically detail the timeline of the incident, root cause and any follow-up actions that are required such as patching etc.

You’re going to get issues so don’t kid yourself that you won’t. Even though outages aren’t great, a service owner can often emerge with a lot of kudos for efficiently managing an incident through to resolution.

Change Management

You have to get this right. Most IT incidents occur as a result of poor change management processes or failure to adhere to such controls. Most enterprises will class failing to follow change process as a disciplinary offence, and I’ve seen it cost people their jobs so whatever you do don’t take any shortcuts, it’s just not worth it.

Your incident management team will probably look after change management across the firm or your business unit so it will be a case of adhering to their guidelines.  Again the approach will depend on the tier of your service, but you’ll probably be scheduling major changes for weekends to minimise the risk if something goes wrong.

One of the key considerations is communication. It’s a good idea to place your power users on the approval chain of the change so they need to sign off that they’re ok with it. If you’ve got a change coming up then make your users aware as early as possible, then send a reminder a week or so from the change date and the day before. Also send them the results of the change (even if it failed) so they’re totally up to speed. This will also inspire confidence in your service.

Change management can be a real pain due to the process and paperwork involved. But it’s a critical aspect of maintaining a solid IT service, perhaps the single most critical item.

Vendor Relations

Keep in touch with your account manager. It’s actually their responsibility to keep the relationship going and a good account manager will be reaching out to you regularly. Don’t be scared to put the pressure on if there are any issues or you feel that you’re not getting the right level of service. It’s their job to keep you sweet. I’ve had good experience with Sarah Bedwell (nee Henselman) in the London office who has been great. Good luck for your move back to the USA Sarah!

Other services in my organisation have really suffered from poor vendor relationships so it really is a critical part of maintaining your robust service.

Managing Perception

So you love your service and you want to get as many people using it as you can. That’s obvious. But it’s not as simple as speaking to people and doing demos. You need to manage the perception of your service and crucially see your engagements through to completion.

Let me give you an example. When I was in full Tableau evangelist mode I was trying to onboard loads of people. So I set a demo up with a particular team and it went really well. They loved it and before I knew it I had requests for individual sessions and help to get started. Unfortunately I did all this at a time when I was mega busy with other work and as a result I wasn’t able to give them as much time as I would have normally. This resulted in the initial enthusiasm dying off and I subsequently found out their manager was giving negative feedback upwards about the whole Tableau service as they felt they had been neglected after being promised so much. That was my fault for choosing the wrong time to approach them. I should have waited until I had the bandwidth to see it through for them. Obviously when I got wind of that perception I managed to circle back to the team and finish the job I started but the damage was done.

Training

Make sure your team is up to date on vendor training, ideally get some of the official certifications that Tableau offer. Not only does this help in terms of technical expertise but crucially it gives confidence to your users that they can rely on your team for true support and engineering (unless the service has a separate engineering team). I’ve seen some cases where a support team doesn’t keep their skills up and before long the users know more about the application than the so called experts. That’s pretty embarrassing.

Community

I’m referring to your internal community of Tableau users here. Here are some tips to help spark that sense of community in your organisation

  • Power users – Identify your key users quickly and build a personal relationship.
  • Lunch & Learn – Have regular get togethers, share how you’re using the tool and any issues.
  • External events – Attend as many roadshows as you can. Bring your power users along.
  • Demos – Tableau are more than happy to come in and do demos. That’s really good way to boost engagement and make the users feel like they are dealing with a vendor that actually cares.
  • Social page – If you’ve got an internal Facebook-style site then spin up a community page. Post your issues, changes, events and news on it regularly. Encourage everyone to contribute.
  • Gamification – Do something cool and fun like Andy Kriebel’s “Viz Cup” at Facebook. a fantastic off the wall idea that I’m sure will be repeated. The subject of gamification is one that’s getting a lot of chat at the moment.
  • Merch – Get some freebies from the vendor. I dish out Tableau badges to my users for a laugh. They seem to like it.
  • Evangelise – If you love Tableau, talk about Tableau. It’s not like Fight Club. I don’t shut up about it. Coffee points, meetings, lunch. I’ve had people give it a go just to get me to be quiet. Then they are hooked. Mwuhhahaha! Make sure you’re able to see your promises through though.

Ok that’s it. If you’ve considered everything in this series of posts then I can pretty much guarantee you’ve got a solid foundation for your enterprise service. There are many different variations of these processes depending on your organisation and there’s really no one right way of doing it. For (a lot) more detail on how to run an IT service check out the Information Technology Infrastructure Library (ITIL) methodology. This framework underpins almost all of what I’ve spoken about in this series.

Happy to expand on any of this. Grab me on @paulbanoub anytime. Thanks also to the fabulous Tableau community for the encouragement I’ve had while compiling this guide.

Cheers, Paul

Tableau as an IT service – Part 3 – Tableau Server Configuration

Go back to Part 1 of this series

Hello all. Thanks for coming back. I’m actually enjoying writing this series of posts. Some crazy hit rates for the first two parts that’s for sure. Most of them probably from my mother but I suppose they all count. Thanks in particular to the people from Tableau that got in touch regarding this series as well as associated Zen Masters across the globe.

Anyway here’s part 3 – Our Tableau Server Configuration

This part focuses on how we set up our Tableau Server application to run on the infrastructure described in Part 2. There may be some overlap and there may be some things I miss – I’m happy to discuss individually. Note there was some evolution of this environment from the initial design and build to what we have today. I’m going to describe the end state rather than the baby steps to get there. Note our service is still fairly immature and a little way off what I would consider a full-spec enterprise deployment but you’ll get the general idea. Tableau Server configuration is a relatively simple affair in comparison to most other enterprise applications I’ve used.

Hardware Specifications

  • Main Server – 4 core CPU, 8GB RAM, 250GB disk space
  • Worker Server, 4 core CPU, 16GB RAM, 250GB disk space

Both servers would share the processing load, in particular the VizQL and dataserver processing. They would both be allocated a significant amount of RAM for snappy rendering and have a large amount of disk space for extract storage. We’ve found this performance to be adequate rather than fast, although that may be due to the fact that the users are hitting the server from a number of geographical locations. Tableau make a number of recommendations to improve server and desktop performance. They also have a really detailed whitepaper on scalability. This sort of professional looking doc is loved by management, even though they’ll never read it.

I must say that this kind of document is a real bonus for users trying to get a service signed off. Scalability is the sort of subject that will be grilled by senior bosses and I’ve been in situations where vendors haven’t provided any guidelines meaning I’ve had to conduct my own scalability tests to satisfy the check point. Having an official whitepaper from Tableau shuts down that question immediately. Great stuff. In my experience the server specifications seem to be less of a factor than poor dashboard design and inefficient calculated field queries. We’ve also seen some dramatic improvements in speed from basic database maintenance such as applying an index to key tables. Tableau have some decent docs on this subject. Obviously this is easier if you’ve got control of the datasource but sometimes you’ll be connecting to an ancient DB thousands of miles away, used by hundreds of users with vague ownership at best so it’s a tricky and risky process to go messing with the configuration.

Authentication

Think I mentioned this in part 2 of this post so I won’t go into this here. It’s pretty key to being able to manage your service as it scales. Don’t even think about local accounts, integrate with Active Directory from the off.

Resilience

The level of resilience depends on a few factors, in particular the budget you have available and also the level of service tier you class your service as. Obviously a Tier 1 service would have a resilient backup server, with the application able to failover seamlessly and quickly, ideally with no interruption to the service for the end user. In the case of our initial Tableau Server deployment we did not configure a failover server as the service was classed as a Tier 3 service. That meant if we had an outage to the application management had accepted (and signed off) that Tableau would be out of action until it was resolved. We made this clear to users as they were onboarded (more about that in part 4) in order to head off any complaints. As our service evolved and grew the usage of Tableau became global and critical to the operations of some business units. Because of this we upped the service tier to Tier 2 and implemented a failover server to allow the service to continue if we had an issue. We didn’t sign up to any SLA agreements as often happens when a new service is commissioned so there was no pressure to provide a four or five nines level of availability, something that is often required by the business. Tableau provides decent docs on configuring your environment for failover and we found it a pretty easy setup process.

Storage

We’ve got big local disks in our Tableau server machines, easily expandable due to the fact we are on the Virtual Machine environment. The key take away from this is that you can’t really predict the disk space requirements of Tableau. You might be fine for ages and then have one user upload a monster extract that fills the entire disk. We tried to hammer this home to the users in terms of best practice and education during the onboarding stage. More about onboarding in part 4 of this series.

Geographical Location

We ran both servers in the same locale. This was down to ease of setup and a necessity to get the project completed (and the service live) as quickly as possible. On reflection we should have considered the location of our potential user base and the fact that adoption of the application was likely to be fast and viral. We certainly underestimated the impact and popularity that Tableau would have. Configuring a system globally has a number of specific challenges.

  • Time – It’s usually easy to get something delivered in your immediate locale. You know the people, you know the process and you can easily wander over to someone’s desk and encourage them to work on your request ahead of other work. If there are any problems then it’s easy to escalate and if someone doesn’t understand your requirements it’s easy to have that face to face discussion to clarify. That’s not always the case when you’re trying to get something installed in a datacenter thousands of miles away from where you are. Processes invariably take longer, and it’s harder to expedite with the personal touch when it’s often someone you’ve never spoken to before that picks up your ticket.
  • Consistency of process – One of my pet hates this. In my organisation I have deployed several global services over the last 4 years. Recently I deployed an enterprise monitoring tool across multiple business areas. We had infrastructure and application masters installed in New York, London, Toronto, Hong Kong, Sydney & Tokyo. Typical tasks involved in the setup were commissioning of the servers, setup of user accounts, filesystem configuration and then installation of the application. Each of those tasks involves a different request process to be carried out by different teams. That’s fine, just needs a bit of organisation on my part. However, the ludicrous fact that each of those processes also varied per region added even more red tape to the process. Unbelievably irritating to the end user to see such a failure to globalize common tasks.

IT management often speak about wanting to “appear like McDonalds” to the end user. You know, you order a Big Mac (Server) in Tokyo and it’s the same process and product as in London or anywhere else. That’s what they always talk about and in reality it’s the absolute polar opposite experience for the user. We completed the monitoring project in 8 months. I think we could have halved that if the setup process has been globally consistent, it really was that much of a handicap.

  • Consistency of technology – This is kind of related to the point above. In my case the standard Windows or Linux VM offerings that you can choose from vary significantly in each region. Same for the available databases (Oracle in New York, Sybase in London), and same for hardware model and numerous other components of the technology stack. Again, adds to delay, inconsistency and user frustration. 
  • Support – Again similar to the two posts above. Some support teams are merged (e.g. Server teams supporting both Windows and Linux with the same process) or they could be completely separate in another region. Some regions might offer 24×7 support, some might not. The monitoring of the infrastructure might also be different (varying disk space thresholds and partitions) and the alerts generated might be handled in contrasting ways. It’s a real throw of the dice in terms of the user experience.

As you can tell the subject of globalisation is one that is a key frustration of mine. In terms of technology and process it’s not that hard to do – but tends to always fall down due to politics, agendas and red tape. It’s disappointing that the ones that suffer are the users and the support teams.

Run As User

networkserviceThis is important. By default Tableau installs and runs as “NTAUTHORITY\NetworkService“. The application will run fine as this user, but we found that the server was not able to access some data sources. In particular we were unable to access UNC paths to Windows Sharepoint environments to monitor Excel spreadsheets. This is a commonly used data source in IT and it was critical that Tableau was able to point to them. In order to allow that we modified the application configuration to run as a domain user, in our case DOMAIN\tableau. That meant Tableau could see the data sources via UNC paths provided the sharepoint was permissioned to allow DOMAIN\tableau to have read access. Tableau make this recommendation in their documentation.

Alerts and Subscriptions

We set up Tableau 8 to send email alerts to our support team (me initially) in the event of a server health issue or extract refresh failure. This setting is useful but needs some refining as it doesn’t seem possible to be able to send emails to different audiences depending on the workbook. I’m not keen on getting woken up for a poorly configured extract failure for one of the user workbooks. Maybe that will come in a future release. For now we just absorbed that headache.

Data Connections

Tableau Server offers a neat feature to control the level of server caching that occurs. I can imagine this being an important tuning point in large deployments. For my case we simply selected “Balanced” and cache for no longer than 1 minute. We’d keep an eye on performance and revisit that setting if required.

DNS Alias

See Part 2 for recommendations on this.

History Tables

Our server was V7 so I didn’t get a chance to try this out initially but will get around to it in the next few weeks. I don’t really need to say too much as the brilliant Russell Christopher has covered it all perfectly. Thanks loads @russch – If you’re not following his blog then you’re missing out. I’m envisioning using history tables and other queries into the Tableau postgres database to produce a live dashboard of the heath of the system. User logons and usage patterns, extract refreshes, hot queries and other metrics all up there for managers and support teams to see. Perfect. Looking forward to having a blast at it.

In Hindsight

Some things that we either didn’t consider or chose not to implement..

  • Load balancer – Our initial service offering didn’t really need this but it’s something I’d certainly consider in the future as the service grows. Some details from the vendor here.

Wishlist

A few things that I’d love to see in future versions of Tableau.

  • Version control – If I’m building an enterprise service I’ll always look for version control. The recent monitoring project I refer to used Subversion to automatically check in configuration files after each modification. It adds that extra level of safety and trackability to changes involving the user xml files. I’d like to see some form of in-built version control in Tableau for those (luckily very rare) occasions where user delete their workbooks not only from the server but also from their local disk.
  • Simpler permissioning of workbooks & projects – Maybe you’re all totally happy with this aspect of Tableau Server? I certainly don’t hear many complaints. Maybe it’s just me but I find permissioning awkward and difficult to keep a proper track of. I’ve ended up just leaving it to the users to permission as they see fit when they publish.
  • Alerting on extract refreshes – 8.1 is a lot better in this respect with the emailing feature but would be nice to see some form of integration with formal alert protocols such as SNMP. I’d also like the ability to configure variable alert destinations based on workbook ownership and other context. It might be a good idea for Tableau to not spend much time on trying to be an alerting tool and instead just dump everything to a log file (with guidance on what patterns to monitor) for integration with a more enterprise class monitoring application like Geneos.
  • Dedicated Sybase connector – I know you can connect to Sybase using the generic ODBC connector in V7 but it seems Sybase was being ignored a little when it comes to the portfolio of connectors. This is no longer an issue now support has been added in V8.

So that’s how we configured our Tableau Server. There’s no one right way to do this and part of the fun is responding to feedback on the service and evolving the offering. You’re not gonna get it right first time. As discussed in part 1 of this series I brought Tableau into the organisation as a small-scale side project and it expanded with popularity. If you’re lucky to have a signed cheque and a go from your managers to implement a service from the start then the approach may be different to mine but ultimately the same considerations apply. This is only part of the journey though, there’s a lot to consider in terms of the actual service you provide to the user base and that will be the subject of the next part in this series.

Next up is Part 4 – Key Operational Considerations