Meet the Penguins!

Posted on June 21, 2017 by

makingof

 

OK here we go. Iron Viz competition time. I don’t viz that much so pleased to dust off Tableau Desktop and have a go. This competition is all about the natural world. A very interesting theme for me being a massive nature fan.

1. The Idea

I love nature. Thinking of a theme I reached back into childhood memories and for some reason I thought of long afternoons with my family at the zoo. Aside from the usual animals we always used to make a beeline for the penguins, something that still happens when I take my own family to the zoo. Everyone loves penguins!

However, I don’t think that many people know just how many different flavours of penguin there are. They live in varied locations, come in a host of different sizes and looks and not all of them live in cold countries. They do all stink though.

So I thought I’d use Penguins of the World as my subject for this viz. And here it is.

Go take a look at the viz!

 

2. Data

I got the data from a single source- https://seaworld.org/en/animal-info/animal-infobooks/penguin/

References to sited data is always good to see

Although I didn’t conduct any lengthy data validation exercise I was given some degree of confidence that the website has a detailed references section, siting the data sources. That’s always good to see and something that is mandatory in scientific papers and such like.

Now there’s tons of penguin data available out there. But I really didn’t have the time to spend days looking for that perfect data source. I also didn’t want to spend days transforming the data before I started vizzing so I settled on this one pretty quickly.  Then it was a copy and paste into Excel and I was off.

Always be on the lookout for good images to incorporate into your viz

One thing I did like was the fact that this page had some cool drawings of each penguin species. That instantly got me thinking of using them as Tableau shapes. It’s always a good idea to be on the lookout for images and drawings that can you can incorporate into your visualisations.

 

3. Viz Design

Now I’m probably not the only person in this competition to be heavily influenced by the master of these kind of visualisations, Sir Jonni of Walker. (@jonni_walker). And with that I thought I’d steal like an artist and try to emulate him.

Key design choices were to use a black background, with plenty of large images and the use of BANs (Big Assed Numbers) as callouts. Things that Jonni does all the time and that really create a visual impact. I also wanted to utilise the penguin images as a “penguin picker” to create some interactivity.

I also wanted a map to be a main feature of the viz as maps are not only informative, but visually striking, highly customisable and also act as a canvas on which to overlay images (in oceans etc.). I had a problem at first with the fact that Antarctica was one of the main locations, and that meant the bottom of the map had a straight edge, which looked ugly. This meant the map would have to meet the bottom of the viz to draw attention away from the abrupt edge.

The IUCN scale

I was pretty pleased with the highlighted IUCN status of each species. The icons looked nice and almost acted as a traffic light theme. Jonni thought they should be greyscale but I overruled him. Pfft – what does he know anyway?

In terms of Tableau content, only a couple of charts to show population, location, height and weight; but that was fine. It didn’t really need anything else.

I also wanted to include a section on famous penguins but it ended up overloading the viz and spoiling the theme. Although it did make me smile. Bonus question – can you name all of these famous penguins?

How many of these famous penguins can you name?

I also considered the use of an embedded YouTube video but decided against it.

I had fun choosing the title font, something that I think can make a huge difference to the viewer if chosen well. Regular fonts were somehow boring, and fonts with penguin characters looked too cluttered. I finally managed to settle on an Austin Powers style font, and then had the idea of alternating the colours to give that penguiny feel. I like it!

In the end I think the final result was ok. However this wasn’t one that I enjoyed. See below.

 

4. Challenges

This was my first viz in a while. I’ve spent the last 3 years knee-deep in Tableau Server and have a crazy busy job building a Tableau Centre of Excellence, supporting thousands of demanding users so I’m the first one to admit I don’t have the Tableau Desktop skills of people like Adam Crahen, Neil Richards, Pooja Gandhi et al.

The standard of skill out there in the community is crazy good. And that really was the main challenge. I found this viz a fairly stressful experience, it made me feel like a newbie all over again, simply because I’d be putting this out there against some stunning competition. I even considered not entering for a while. But hey, that’s not what the Tableau community is all about so I thought I’d have a go at it.

As mentioned earlier, I was deliberately trying to emulate Jonni’s style. Now that proved to be pretty difficult. I managed to create something reasonable, and fairly quickly, and began thinking to myself that hey this is a piece of cake, Jonni who?? But then it got harder. My ideas began to dry up and I found myself staring at an okay-ish viz but being unable to take that next step to make it better. Felt like vizzers block.

And that’s when I realised that the people who create these REALLY good vizzes have a lot of inherent natural skills and imagination that folks like me lack. So I gave Jonni a call and asked his advice. He came back with a number of suggestions, none of them earth-shattering, but much more subtle and delicate. Making the map larger, bringing highlight colours out from the penguin plumage were a couple of suggestions that made a huge difference to the impact of the viz. My point here is that the real geniuses of visual design have these thoughts occur naturally and without significant effort, folks like me have to learn them, or at least work a little harder than some others.

But hey, IronViz (and any vizzing) is all about learning. So I’m good with that.

Another challenge was that the Tableau part of this was pretty easy. That’s obviously great and what we want from our favourite application, but in terms of this viz I spent more time in image manipulation tools than in Tableau. And that really did detract from enjoyment. Come on Tableau! Make it harder for us to complete our vizzes!

 

5. Analysis & Story

So what can we take from this story? Here are some of the key observations that Tableau has allowed me to glean from the dataset.

  • All penguins live in the southern hemisphere, and some in hot countries
  • There are some seriously big populations, although some are endangered
  • They range from massive to teeny tiny
  • Main predators are Leopard Seals & Sea Lions
  • There are not one, but two penguin days in the calendar

So that’s it. I hope you enjoy the visualisation. If you do then please consider voting for me in the IronViz competition. And thanks to Jonni Walker for providing advice for this viz. Top man.

Good luck to all the other entries this year. Especially blinders like this from Ken Flerlage. – The Killing Fields – Viz / Blog.

Hmm. After all that writing I could do with a chocolate biscuit. Now which one……

Regards, Paul

Posted in How to..., Top Tips | Tagged , , , , , , , , , , , , | 1 Comment

What we got wrong building our Tableau service

Posted on May 2, 2017 by

Hi

I spend a lot of time talking to other companies about how to put together a decent Tableau Centre of Excellence. And that’s cool. I love to help people and I also learn a ton from all the other great setups out there.

But that also gets me thinking. We are far from perfect. And we made a whole lot of mistakes on the way to building our service. So instead of talking about the good stuff we did, I thought I’d highlight what I think we got wrong.

So in no particular order of importance, here are our top mistakes

Onboarding

So you’ve done the demo, shown the capabilities of Tableau and your audience is wowed. They want it and they want it NOW! So how do you get them from wanting to having?

Well we didn’t do so well. Our purchasing is handled by another team and the process is fairly complicated. Our mistake was not building a solid enough relationship with the purchasing team early on and making an effort to understand the points of the process that could be improved. We kind of just let them get on with it when we could have offered more assistance. This meant that on occasion users had to wait up to 2 months before they actually got their licence! And sometimes that cost us users, who gave up during the process.

Eventually we got together and helped the purchasing team out. And now things are a lot better. My advice – if you see a team you have a dependency on are struggling, then speak to them and help them out. Sounds obvious, but on this occasion we didn’t.

Setting Expectations

I spend a lot of time talking to users, often at a senior level, asking them to feedback on their experience with Tableau. On the whole it’s great stuff that comes back, but occasionally I used to get surprised with users informing me that their experience was poor. Tableau didn’t do what they wanted, it was inflexible and hard to use.

That didn’t sound like the Tableau I know and love. So I did some more research and it turned out that the users who didn’t like Tableau were trying to get it to do something that it wasn’t designed for. Now that’s fine if you’re Allan Walker or Noah Salvaterra but most of my users aren’t that level – in fact few people are. And of course that meant the users were getting frustrated.

I think the comment that really resonated was “Tableau? That’s Excel online isn’t it?”. Er – NO IT ISN’T!

The problem was obvious. Users thought Tableau was something it wasn’t. And naturally they would get a degraded experience. So we created a document that clearly states what Tableau is good at, and what it isn’t good at. We make users read this doc when they sign up for the service so they know exactly what they are getting.

This was very successful and really cut down the instances of poor feedback.

Some related posts on this subject from Dan Murray, Matt Francis & Peter Gilks

Training from the Get-Go

I’m a self-learner. So is my team, and most of the people I work with. I’m sure you are as well, that’s why you’re here reading this. And as a result I expect others to be the same. And for the self-learner, the world of Tableau is great. Tons of bloggers, forums, help articles, Tweeters, online videos and all sorts of quality learning materials. It really is one of the strengths of Tableau IMHO.

So we created guides & 101 pages etc and told users to go and help themselves. And to be fair, some of them did. But not as many as we’d have liked. And as a result we got tons of newbie questions and consequently poor quality content on our server.

About 8 months into the service we decided to implement a structured, instructor led training programme, amongst other training initiatives. More details here. Much of the syllabus is covered by Tableau’s own online videos and other resources but for some reason, new users really responded to this structured course. As a result we saw a corresponding improvement in the quality of published content and a reduction in the basic level questions to the team. In hindsight we should have implemented this right at the start, rather than assuming everyone would be keen to self-learn.

Commercials

Much as I love Tableau, their commercial operation isn’t the most flexible. That’s not just my opinion, it has been mentioned in Gartner reports. As a result we’ve struggled to negotiate the most competitive packages that we could have done. There have been some valid reasons for that but I don’t think we put as much effort into resolving the related issues as we should have.

And that’s important. I don’t want to get the best price for the sake of it, in big enterprises there are a lot of competitive threats from other tools. If Tableau isn’t competitive with pricing then the people that make the technology decisions will be perfectly happy to rip it out and replace it with something else. Not what anyone wants.

Again much of this is down to relationship building. As our relationship with Tableau has grown, we’ve seen improvement in this area.

Gamification

I’m a big fan of gamification. It can really boost engagement and turn a good initiative into an awesome one. We’ve had a couple of half-hearted stabs at competitions, to try and get things going but there is so much more we can do. Hackathons, viz games, competitions and internal Iron Viz – we’ve got a ton of ideas but never made it happen yet.

I think that’s partially down to the wide geographical spread of my user base. It would be really hard to get people to show up in a particular location. Users are also insanely busy, would I be able to round up enough people to make it worthwhile? So lots of excuses and not a lot of achievement in this area. I’m open to ideas if you’ve got this one nailed.

That’s it for now. There may well be a part 2 to this as I’m sure there are more things that will come to mind as I cry myself to sleep thinking what a mess we’ve made of it all… 🙂

Thanks to the awesome Matt Francis for inspiring this post.

Happy vizzing, VN

Posted in Tableau as an IT service | Tagged , , , , , , , , , , , , , , , , , , , | 4 Comments

The Three Levels of Tableau Support

Posted on January 12, 2017 by

Hi all,

Let’s talk a bit more about how to build a top Tableau support team. This post focuses on the support my team provides to our user base. At the moment we have just over 1000 Tableau Desktop users, and approximately 8000 active users on the server every month – that’s a lot of demand for our services.

Now users can be a pretty demanding bunch, with myriad questions, queries and problems. And we are busy. So how do we ensure that users get the level of support they need? Well we provide 3 levels of main support, with the objective being to ensure that the type of user query / issue is directed to a channel that gives it the appropriate level of attention. This ensures an efficient use of my team’s valuable time, and critically it cuts down the traffic to our email inbox which is always a good thing.

screen-shot-2016-11-22-at-14-45-32

Some of the support options for our users

Level 1 – Man down!

Red alert! Something is busted and it needs to be looked at now! For this we need any incident to be logged in a trouble ticket system, with appropriate priority and detail. We use Service Now for this (many other tools available).

So if users think Tableau is broken or they need some immediate help then they log a ticket. This is mandatory. We need to track and log the progress, and the data is audited regularly. No ticket, no fix. We obviously don’t wait stubbornly for the ticket though, if there’s a big issue we investigate while the incident is logged.

Once the ticket is logged it flows through our regular support flow. First my Level 1 team will take a look and see if there’s an easy fix. If they can’t fix it then it’s an escalation to my more skilled Level 2 team, and then a potential escalation to my main Level 3 team for the trickier issues. There may be a future post coming about effective incident management, so I won’t go into detail here.

Some users don’t like us mandating that they raise an incident ticket. But it’s the only way to ensure traceability of problems.

Level 2 – It can wait

Sometimes users have problems or requests for assistance that are not so time sensitive. Maybe a development dashboard has broken, or someone needs help from the team to perfect that Pareto chart, or hey – maybe they just wanna talk about how much they love Tableau (it happens!)?

screen-shot-2016-11-22-at-14-48-33

Book your appointment with a Tableau Dr.

That’s where a Tableau Dr. Session is needed. We dedicate 3 half days a week to Tableau Dr. Sessions. Users log onto our community page and can book their session from a list of available slots. If the next slot is in a couple of days then they have to wait to be seen. Providing this structure to the sessions is critical as it allows my team to keep control. Before we implemented the structured sessions we were getting peppered with do-it-now requests for Dr. Sessions. That meant my team was context switching all over the place and other projects were being impacted.

Providing structure also makes users understand this is a finite resource and thus they are more appreciative of this dedicated time with my Tableau experts.

 

Level 3 – Let’s talk about Tableau, baby

Next level of support is for general chat. Could be a question about functionality, or a point about performance, or a geeky joke, or someone just wanting to ask a question about our upgrade strategy – could be anything really.

 

That’s where our Lync Group Chat comes in. We’ve generally got a couple of hundred users on the chat channel at any one time so it’s a decent forum for such questions and banter. It’s great for my support team to see a question get asked, and then before we have a chance to pick it up, another user has provided the answer – a self healing community – IT support nirvana!

screen-shot-2016-11-22-at-14-45-08

Wanna chat Tableau? Use our Group Chat

 

What’s in it for me?

These support options ensure that each query gets an appropriate response. If it has all hit the fan, then we act quickly. If it needs more care and detail, then we book that time, and if it just needs someone to talk to then we’ve got a community of people ready to give that data hug. It also means we get hardly any emails. And email is a dreadful means of logging an issue, as there’s no traceability or feedback. Users only get annoyed when they feel a query is being ignored, and ensuring the correct channel for a query means users get feedback as appropriate and aren’t left wondering where that email question went to.

Also my support team can plan their work and aren’t constantly context switching, one of the biggest enemies of productivity.

So that’s it. Pretty simple to implement but mightily effective. As always, ping me if you want a more detailed run-through.

Happy vizzing, Paul

Posted in Tableau as an IT service | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 3 Comments

Tableau on Tour Keynote Speakers – Some Suggestions

Posted on December 13, 2016 by

This gallery contains 32 photos.

Hi all, I love the Tableau Conference. But I also have a lot of fun at the smaller “Tableau On Tour” events. In particular I love the keynote speeches. We’ve had some crackers recently, with particular recent favourites being Tim … Continue reading →

Gallery | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Empowering Your Tableau Users With Makeovers & Proactive Support

Posted on November 21, 2016 by

Hi all,

More on building that dream Tableau Centre of Excellence function. I’ve previously posted about how to structure your support team and ways to build user engagement with “Tableau Champions”, this post focuses on how you can use Tableau’s introspection capabilities to deliver a more proactive support function.

What is proactivity?

The traditional definition of proactive is as follows.  To me it means means seeing into the future and Screen Shot 2016-08-07 at 21.21.00getting to an issue before it even happens. In the world of IT Support, proactivity really is the Holy Grail, meaning the difference between a good support function and an amazing one. But it’s super-hard to achieve, especially in the complex enterprise level setups that have multiple break points. You can almost never prevent something from breaking, no matter how good your monitoring is.

What you can do is add some proactivity into the way your team operates by identifying when your users are not getting the best from your service. In Tableau Server world we have the ability to spot the following and much more.

  • Slow Tableau visualisations
  • Consistently failing extracts
  • Stale content

I won’t go into how to achieve this, it’s the subject of a future post. But I’ll point you in the direction of these 2 posts that should get you on the way. Go check out Custom Admin Views by Mark Jackson and Why are my Extracts Failing – by Matt Francis.

I get my team to scan our admin views, to identify those users that in our opinion are not getting the best experience they can from Tableau. If we see someone who might be experiencing consistently slow visualisations, or have regularly failing extracts then we give them a call. Often the users won’t even have a complaint. But our message is “We think you’re not getting the best experience possible, and we want to make that happen”.

screen-shot-2016-11-17-at-21-17-13The initial reaction is often surprise. “I’m ok, I didn’t raise an issue” will be a common response. But then once we’ve worked with the user, and improved their experience, you’ll find they are blown away. You may even get a call from their management!

You’ll find this kind of service is very rare in most organisations so if you can deliver it, even sporadically, then you’ll be regarded very highly.

Makeovers

This is pretty simple, if a little time-consuming. Browse the Tableau content on your server. Spot something that doesn’t look great – it might be slow, not compliant with your best practices, or just fugly. Download that content, and give it a makeover. Make it look great, maybe add some improved functionality, make it nail best practices.

This is one of my team’s favourite activities due to the reaction of the user / client. They LOVE it. It really creates a sense of engagement, the user feels that your team actually really cares about them. We’ve also had our Tableau Champions participate in Makeovers which is even better as it saves my team some cycles.

Be careful though, some user content might be confidential and the user may not appreciate an admin poking around in their data. Also, remember that by doing this you are implying a criticism of their work, so handle the communication with care and sensitivity.

Also ensure that you don’t just change stuff and then drop it back on their laps. In a self-serve model like mine users develop and support their own content so it is crucial the user knows what you’ve changed, how you’ve changed it and what benefits you feel the modification brings. Pull them and their manager into a call, run through what you’ve done and then hand it back over to them to run with it.

These have been very successful in my organisation. Users truly appreciate the help and my team has fun doing it.

So there you are, a couple of tips for adding that gloss to your Tableau support service.

Cheers, Paul

Posted in Tableau as an IT service, Top Tips | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 2 Comments

Building user engagement with Tableau Champions

Posted on June 2, 2016 by

Hi all,

More on building an enterprise Tableau Centre of Excellence. That’s pretty much all I know about hence why I seem to be writing about it a lot…

This is a short post about an initiative that is proving to be pretty successful at my organisation, we call it Tableau Champions.

images

We are the Champions!

We’ve based this loosely on Tableau’s own Zen Master initiative. For those that don’t know, Zen Master is effectively a title awarded to members of the community on a yearly basis. For more information see here – http://www.tableau.com/ZenMasters

 

What makes a Tableau Champion?

We award the Champions badge to users that demonstrate

  • Passion & enthusiasm for Tableau & data visualisation
  • Support of the Agile BI service at my organisation
  • Skils in Tableau & visual analytics
  • Willingness to share & assist other Tableau users
  • Involvement in the Agile BI community

Even amongst a huge user base like I have, it is easy to spot users that demonstrate these characteristics. They will become your trusted advisors, providing great feedback and helping you iron out the bumps in your service.

 

What’s in it for a Champion?

Here’s what my team does to help Champions

  • Build Tableau skills & contacts
  • Increase internal profile across the org & gain stature as a Tableau SME
  • Increase external profile
  • Exposure to extra product information & roadmaps
  • Contribution to the development of the Agile BI service
  • Great collaboration opportunities across the firm

 

 What’s in it for my service?

And in return Champions help us by

  • Makeovers & dissemination of Best Practices
  • Publicising events & webinars
  • Blogging on Agile BI community site
  • Host local user groups
  • Champions help local users evolve Tableau skills
  • Driving better understanding of visual analytics & Tableau

 

So it’s a mutually beneficial scheme, with Champions effectively acting as an extension on my own team. Win and indeed – win.

One thing I noticed was the way the Champions initiative immediately started to raise the bar in terms of user interaction with Tableau at my org. No sooner had I posted the first blog announcing our initial Champions, then I had multiple emails from other users saying “I want to be a Champion”, “What do I need to do to get this recognition?”. I could even tell that some users were a little miffed not to have been selected. I then saw these users upping their game, posting more, interacting more, trying to be noticed. We’ve seen this with the Zen Master scheme eliciting exactly this kind of response from the external community.

So there you have it. We love to empower our users. And we love to reward those users that have become hooked on Tableau like we have.

Cheers, Paul

Posted in Top Tips | Tagged , , , , , , , , , , , , , , | 2 Comments

Tableau Server – all about the… Backgrounder

Posted on May 5, 2016 by

Hello everyone.

You may know me as a Tableau Centre of Excellence manager. That can involve a lot of paperclip pushing skills, with the real work being done by my excellent team (thx @jakesviz & The Information Lab). But I do try and get down and dirty with my lovely Tableau Server environment to keep my skills fresh. Obviously I don’t mess with it – @jakesviz gets pretty protective about his Server.

This series of posts is my attempt to shed some light on the internals of Server. Note there are many more experts in this field than me (Craig Bloodworth, Mark Jackson, Jen Vaughan, Tamas Foldi, Mike Roberts, Angie Greenhaw – to name but a few) so please do comment if anything is incorrect here. Maybe you guys could help me evolve this post?

What is the backgrounder?

The backgrounder is a process that runs as part of the Tableau Server application. As the name suggests, it handles background tasks such as refreshing extracts, running subscriptions and also processes tasks initiated from tabcmd.

Here are the backgrounder processes. The .exe file and the .war file. The WAR file is a Web Application Archive, and contains all the necessary components and resources needed for a Web Application such as Backgrounder.

On a clustered environment you’ll find these files in D:\Program Files\Tableau\Tableau Server\worker.1\bin (may vary slightly with your installation).

28-01-2016 14-07-52

Other files related to the backgrounder.

Template (.templ) files – These files TBD

28-01-2016 14-13-09

There are also a few .rb files in D:\Program Files\Tableau\Tableau Server\worker.1\tabmigrate\db\migrate.

And we also have a .properties file which contains all the config entries relevant to the backgrounder. It also has almost all of the other stuff that you’d find in the main workgroup.yml file which is odd. I’d  have expected it to be just the backgrounder config.

backprops

Here is the location of the backgrounder log files

backlogs

Here are 2 instances of backgrounder.exe running on my server (from Task Manager).

28-01-2016 14-01-08

 

Can I mess with it?

Backgrounder can be configured. There are several settings present in both workgroup.yml and backgrounder.properties. Workgroup.yml is the master config file, and it populates the backgrounder.properties (and other .properties files) when a ‘tabadmin configure’ is run.

I don’t know what all of these do (yet) and the only one I’ve ever edited is ‘backgrounder.extra_timeout_in_seconds‘ which sets the max time in seconds that a backgrounder session can run for. Tableau kills off the session if this threshold is reached. Useful for forcing users to optimise their extract times!

I also pay attention to the ‘backgrounder.vmopts‘ parameter, as this defines the size of the java heap space for this component. All components have a vmopts setting and I’ve had to increase them on occasion due to out of memory problems.

You may also want to change the ‘backgrounder.log.level‘ if you need more debug info, although Tableau logs are chatty enough for me.

If there’s a golden parameter in this lot that you get value from then let me know in the comments.

backgrounder.deploy.dir: D:/Program Files/Tableau/Tableau Server/data/tabsvc/backgrounder
backgrounder.external_cache.concurrency_limit: 10
backgrounder.external_cache.enabled: true
backgrounder.external_cache.num_connections: 1
backgrounder.external_native_query_cache_disable: true
backgrounder.extra_timeout_in_seconds: 1800
backgrounder.failure_threshold_for_run_prevention: -1
backgrounder.jdbc.wg.connections: 8
backgrounder.jdbc.wg.idle_connections: 4
backgrounder.log.dir: D:/Program Files/Tableau/Tableau Server/data/tabsvc/logs/backgrounder
backgrounder.log.level: info
backgrounder.native_api.log.level: info
backgrounder.out_of_date_schedule_minutes: 240
backgrounder.ping_dataengine.millis_to_wait: 5000
backgrounder.ping_dataengine.num_retries: 24
backgrounder.ping_services.num_retries: 10000000
backgrounder.ping_services.time_to_wait: 5000
backgrounder.purge_directories.directories: ""
backgrounder.querylimit: 7200
backgrounder.restart_interval_in_minutes: 480
backgrounder.restrict_serial_collections_to_site_level: true
backgrounder.search_index_verification.enabled: true
backgrounder.sheet_image_api.max_age: 240
backgrounder.sleep_interval: 10
backgrounder.sort_jobs_by_run_time_history_observable_hours: -1
backgrounder.sort_jobs_by_type_schedule_boundary_heuristics_milliSeconds: -1
backgrounder.timeout_tasks: refresh_extracts, increment_extracts, subscription_notify, single_subscription_notify
backgrounder.tomcat.threads: 4
backgrounder.urlprefix: backgrounder
backgrounder.vmopts: -XX:+UseConcMarkSweepGC -Xmx512m

Note that Tableau Support don’t like you to edit config files manually, they recommend that you use the tabadmin set commands to change any parameters. They might have to change that recommendation when we see Tableau Server on Linux.

For more about .templ, Ruby & properties files check out this from Tamas Foldi.

 

What are the problems with the backgrounder?

Here are some of the things that can be problematic with the backgrounder.

  • Single Threaded – This means the backgrounder process can only run one thread at a time, a thread being a set of executable instructions that a process can perform. The upshot of this is that your backgrounder works through a queue of tasks one-by-one.
  • Latency – Due to the single threaded nature of backgrounder, you may see delays or ‘latency’. For example, if you have one backgrounder, and 2 tasks for it to perform at 2am, then task 2 will have to wait until task 1 has finished. If task 1 takes an hour then task 2 won’t start until 3am. This can be annoying for users that expect their data to be refreshed by a certain time.
  • Resource Intensive – The backgrounder can consume a significant amount of processing power (CPU) and input / output (I/O) on your server. This is dependent on the type of task it is performing. It’s not uncommon to see a backgrounder node consuming 100% CPU.
  • Other Stuff – The backgrounder process also does other stuff on Tableau Server that isn’t concerned with extract refreshes. For example – reaping extracts, checking disk space, synching Active Directory groups, rebuilding the search index etc. Bear that in mind when building your system. In reality these tasks don’t take up too much resource but they do take some and you should be aware.

 

Isolating the backgrounders

A common configuration in a clustered environment is to dedicate one of your worker nodes to the backgrounder processes. This means you can dial up the number of backgrounders and let them do their stuff without worrying about any impact on other processes. This is one of the most common performance recommendations from Tableau support.

You can also get a lot of info out of the Postgres DB relating to backgrounder usage and performance. Nelson Davis has posted a guide to getting started here.

 

Improvements I’d like to see

Ok so here are a bunch of improvements I’d like to see to the topic of backgrounders and extract management. I know some of this will drop in upcoming releases and some of these problems have been solved with custom solutions at some customer sites.

Alerting – Tableau Server doesn’t alert (email / IM / SMS) when a task fails. This means you’ll need to set up external monitoring to detect issues. I know Tableau are on this though so expect to see it in an upcoming version. Some people in the community have also coded their own solutions to this problem but it really should be native functionality.

Control per site – We segregate our dev / test / prod user environments using sites , all on the same server. We run 8 backgrounder processes on that server, which are shared across the tasks on all sites. As an administrator I’d really like to be able to bind backgrounder processes to specific sites. For example, 2 backgrounders on each of dev & test sites, then the other 4 dedicated to the production site. That would ensure production tasks always have enough resource to be able to execute on time.

Control per process – I’d like to be able to stop / pause / mess with individual backgrounder processes easily. It is possible – see this from Toby Erkson, but it would be good to have this as part of an administrator console or something.

Control per type / size / pattern of extracts – It would be good if I could dedicate specific backgrounder processes to particular extracts based on their characteristics. In particular I’d like to allocate one backgrounder to all the extracts that take less than 1 minute to complete. Or even use this to reward users that show diligence with their extract management by dynamically prioritizing incremental refreshes or extracts that have a low failure rate.

Better metrics – I’d like to see exactly how much CPU is taken up by a particular backgrounder process or task / schedule or per project. This would be useful for chargeback.

Dynamic reprioritisation – I love all my users. But in particular the ones that take good care of their extract refreshes. I’d like Tableau to be able to dynamically increase the priority of tasks that complete quickly, are incremental and that have a low failure rate. The message being, if you want your stuff to get the best slice of available resource then help us out with best practice.65

Disable run now – We’ve had some issues with the “run now” option that allows users to kick off an extract refresh on-demand using the UI. In particular we’ve seen some trigger-happy users bring our server down by hammering on the run now option multiple times. I’d like to disable that or maybe throttle it somehow.

Better guidelines from Tableau – The documentation from Tableau isn’t great in this area.

05-05-2016 13-09-11

I have an 8 core 128GB server and  run 8 backgrounders with no capacity issues. And I know of other organisations running way more than that. According to this doc I should be running between 2-4. That would be some serious under-utilization of the server. I’d like to see some clearer recommendations, maybe taking into account the variety of use cases that I’ve seen in other big enterprise deployments.

 

OK that’s all for now. This post could have been more detailed but I figure that I’ll get some valuable inputs from the community that will help me expand it. Actually in the time I was writing this, Mike Roberts was doing the same – check his post for some excellent info.

Cheers, Paul

 

Posted in How to..., Tableau as an IT service | Tagged , , , , , , , , , , , , , , , , , , , | 8 Comments