MID Server ECC Queue Aleck Lin 1

um my name is Alec and I don’t know what Debbie has told you about me don’t believe don’t believe any of it just kidding um so I’ve been in service now for about five and half years so coming up on six I I am been working on discovery for pretty much the most of the time I mean there’s there’s some offshoots brought a few projects here and there so anyways today I’m gonna talk about miss server and and discovery and and a huge portion of this will be focused on the the majority this this book boot camp session we focused on the the architecture of the miss server and instance and and then and I’ll talk a little bit about discovery toward the end just so that that you understand what it is okay so before we get started though there’s a couple of things I want to sort of sort out so you guys all have your eclipse instance or if not you should have a local it you should have a instance on in the cloud right okay and then can you guys have you guys to enable discovery not yet okay awesome so um okay cool so you haven’t please navigate to the plugins on module and then a activate discovery so what it looks like is you go to plugins and then search for discovery and then you should be able to just right-click on that that and say activate of great on discovery so if you haven’t done that so that that takes a little while so I want to get that out of the way so we don’t end up coming back and if you’re now seeing discovery in the list of the plugins that that means that you do not have mayn’t access so if you do not let me know we need to figure it out okay for those of you who are a little more advanced in if you already activate the discovery there’s something else I want you to do there’s a there’s a prepackaged miss server that’s already ready to go to be installed so I would like you to basically download it first so just follow the instruction on the slide on a slideshow basically SCP the MIT OSX zip eve maybe I could help you activate the plug-in if you if you can send me the chat what the what the instance name is yeah so so to sort of explain a little bit there there are only actually about two plugins that require mains access their discovery and orchestration and that is because they are the customer our customers have to pay extra for discovery or orchestration so that’s why they typically have the customer request the deactivation the plug-in after they bought it of course yeah so if you here’s what you can do open up a terminal if you don’t have it you can do a command spacebar and then you can find your terminal program once you’re there you can simply just follow the command that I had on the screen so in this case so well one thing that I want you to know is that make sure you know where you download it too because we’re gonna have to come back to it and so in the past the people the app people download it and then they’d then forget where they put it so typically if you open our terminal it’s just in the home directory so you can paste it and you can download it it should shouldn’t be shouldn’t have any problem so it should look like that okay the password is a this disco okay so let me get started so we talked about miss server and the ACC q so I know I know many of you have you know varying degrees of familiarity with our instance so I’m gonna you know try to help you understand what it is even if you never heard of I’ve heard of it so mister obviously the first question is what was the mister and I’m sorry about you already know this but basically it’s a it’s it’s conceptually it’s it’s if the instance hands and

eyeballs right so it’s a it it helped us it enables us to reach out to our customers network and being able to get information or to act execute certain commands it’s basically a Java software application can run on Windows Linux or in this case Mac OS Vishal you only support Windows and Linux for customers internally we could use go as it’s typically installed by a by the customer on a connected on a host that’s connected to their network obviously and the whole point here is that we don’t want our customers to punch any firewall any holes in the firewall to allow our instance to talk to the miss server so basically the Miss server always makes an outbound HTTP request to our instance okay so that’s what that so basically there’s a there’s a there’s a thread that basically pulls every 15 seconds to say hey you know do I have something to do on the instance also it uh it it upgrades automatically with the instance so as the instance moves from say Calgary to Dublin to Eureka to Fuji the miss server automatically will look at the version is supposed to be on and move upgrade itself okay so what does it do it basically is controlled through the instance the instance has the is typically what we call the brain of the operation right so the miss server says connects to the instance they hate do I do you always do something and then a gig and if the instance has something the mystery will grab that and say okay I’m gonna execute whatever you want me to do it’s there are a lot of protocols that we’ve built into the mid server it’s a capable of talking with SSH SNMP WM is an HTTP so breast powershell what-have-you so it’s pretty you know that it’s it’s quite a quite quite a bit of I guess versatility the miserable brings to the sort of table I mean obviously there’s more we can do depending on the particular integration and and things it is used for imports discovery orchestration and integrations so it’s not just a typically when in in our company a lot of people think that miss server they kind of sort of think of discovery associated with it but but it’s actually very much used by orchestration and our integration team as well when it comes to integrating with a third-party software or for orchestration to actually execute commands to do something currently it is a stateless meaning that you can install as many miss servers you want to and they do not keep any stay on there and and when you restart the Miss server any time it will just you know you would pretty much be back to where it’s supposed to be so because dean like I said the instance is brain of the operation so anything that that the Miss nomer needs to do comes from the instance right so in the mid server say it’s in the middle of executing something and it dies that’s not a problem it you know you restart the Miss server you figure out what’s wrong with it and restart it and it’s gonna pick up right where it left off okay so this is a visual diagram of what that looks like right you know in the architecture so you can see the Miss server it’s within our customers Network and it basically goes through it could go through a proxy and and goes through the customers firewall and then hitting our instance that’s on the internet somewhere right so as you can see a Miss server you can see the arrow on it communicates to the instance and they also come indicates to whatever internal devices there are within that network so typically they’re the installation it’s very fairly simple there’s a module called miss server that that comes straight out of the box as soon as you have the instance in enable the you can download it click into the download page it’ll allow you to download a zip file we support 32-bit 64-bit miss servers if you’re interested in as a developer you can run it directly within Eclipse and that’s what you know a lot of us to run on the automation team that’s what we do obviously to develop and troubleshoot the mid server and so you could it’s basically just need to import a mid project and from funk it and then you can just launch it after you configure it but for the purpose of our exercise I already prepackaged aim the server which you would download it so

later on we’ll we’ll unpack it and and install it okay so let’s talk about the ECC queue ec c stands for external communications channel basically the MIS server communicates with the instance through the ECC queue now the ECC queue is nothing more than just a table currently it’s a table that that says missus or go do something and when the member does something is send something back and back into it so we can think of the the ECC queue as sort of as a a Grand Central Station with you well you know moving things along right what were it collects everything everything comes in and everything goes out essentially and and uh the the miss reviews so web services to query and insert records that should be fairly straightforward and then each record in the ECC queue represents a single self-contained unit of work it doesn’t serve it needs to perform so as I said you know there there’s a concept of of the Miss server output record that represents a unit of work that a mid server can pick up and do something and once the mid server finished that task it then sends back the payload back to the ECC queue table as a as an input record okay what we’ll go through that a little bit more later but that’s that’s a general idea so here’s a visualization of what that looks like right so you got your instance on the upper left-hand corner and you gave it your EC CQ table on the instance so very first thing it’s obviously if you have a chunk of work from the mid server that’s specified in the ACC Q then the next thing that’s gonna happen is the mid server every 15 seconds it’s gonna come in making soap requests to the instance and say hey do you have anything for me so they the instance say hey oh I do have something for you and I want you to ssh to this particular IP address and i want you to run a command in this case a you name – SP now the mid server grabs that command and it says oh cool I’m gonna I need to run an SSH command against this particular IP address and it launches that come in against it and obviously the target it’s gonna respond if you’re able to connect to it and in this case the the the targets can respond oh I’m a Linux I 686 so the MIS server gets that and then package that up into an XML file and then sends that back into the ACC q okay I think that’s that’s pretty straightforward right any questions yeah as I go very good question good question the the the the instance actually specified in a in a table called credentials table on the instance and when when the MIS server starts up one of the things it does is it communicates to the instance and and and downloads a list of credentials and it stores in the member it stores it in memory on the mid server until until my serve yeah that’s right is oh yes excited the caviar act slightly correct that’s likely the miserable communicates to the ECC queue every 15 seconds it is not configurable at the moment it is it is just 15 seconds what is there another question yeah I mean that’s something that we have thought about and I think they’re coming you handy in certain cases we just I don’t think we have any specific use case at the moment to to to allow people to do that I mean I we also don’t want people to Jack himself up but you know changing a tooth I don’t want say a thousand seconds you know which would be too slow or every second would be too fast okay any other question good no no no no there’s a slight clock progression today so the the mr. Kirk queries for work every 15 seconds but the the the once if that’s the work and has has the payload ready that’s actually a different thread

that sends back to the instance and after it has actually run every second so it meaning that that if there’s anything to be sent that’s right it’s gonna it’s gonna be constantly sending things back to the instance sorry say is it part of a question it could be an SSH command it could be a PowerShell script or you know I say just quit for that matter I can do SNMP you can do JDBC queries and things like that so I mean later on we’ll we’ll go through an exercise where you actually construct a unit of work for the MS server and make it do something so you can so hopefully you’ll get a better idea what I look what I feel what it looks like okay cool all right so oh the last thing I mentioned was so once the the payload or the result gets back into the instance in on the ECC queue at that point there’s going to be a some sort of business rule that triggers say that that would then follow on to various processes to say hey you know what do you want to do with this particular payload right so in in case in a in the case of discovery it’s gonna say okay this pretty good payload it’s come coming from our discovery application and so discovery code to go through something with this data right so is you can imagine in in the case of discovery you could be we’re running a say and what PS you know – you f – D F and then any you’re getting a list of processes and discovery will try to parse that information and storing this to the database in a case of runbook it could be sorry not run book with orchestration sorry I keep saying run book because orchestration use of coal run book so I’m just just old habits die hard okay so in okay so constriction you could be executing a piece of script to say you know I’m gonna update certain configuration record things like that so so they have a different path – once the payload comes in different business business rules will trigger to do that all right so let me get into a little bit about what a what is a unit of work on ACC Q and what it looks like so here’s a ECC queue form for for a particular record so on here this is just a where was that okay so this is a form of the ECC queue table and as you can see on here it’s basically specifying things like hey which the agent field specifies which miss sugar you wanna this this particular unit of work is meant for the topic talks about what kind of command it’s a what kind of probe if you well what kind of come in it needs to be so as I said earlier it could be a SSA she could be PowerShell it could be JDBC what have you the name here the name field here is a it sort of defined depending on the particular type of probe or work it’s being done so in in a simple case of SSH command the name field can be used to specify the actual command you were going to run so on here you see it’s doing a cat slash procs flashman info to get memory information the source field signifies the IP address the MIS server needs to go after so in this case is talking about specifying particular one response to don’t worry about that the Q the Q field basically says signifies this is an output on Impa Q so in this case you’re seeing an output ECC queue record so you know that this is meant for the MS server to pick up okay the State field says you know it’s ready or processing or processed and then everything else is sort of just peripheral data that that you sort of for your information but the point is that there there are five sort of or six fields that are most important the agent topic name source at the cue state you know those needs to be specified properly in order for the missile to pick up so here’s an example of what happens after the missile has picked it up and executed the command and sent it back to the instance so in this case everything almost looks the same almost you know your agent your topic your name your source pretty much looks the same you notice the cue is different the cue is an input so you know that this record is came back from the Miss server and then in that case you can look at the payload field then you will see that there’s actually results in there right

so this is the result after I have run or rather after the Miss server has run cat /proc / mini info so it just give you gives you back the memory information and what not make sense so far right okay all right so now we’re gonna actually get hands-on right so it’s cool to know what it is but I think it’s been cooler if we can actually make it work and you can can’t see how that works so the first thing first I want you to install the Miss server on your on your computer so earlier you have probably heard downloaded the the Miss server on to view a lot on to your laptop so basically the next step I want you to do is basically unzip that mitt that zip file that you have downloaded and you can unzip it either you know find out where it is you can unzip either by click double-click on a zip file or you can open up a terminal and do just unzip followed by the filename okay so um sorry um before you uh start running it there’s something else and we need to configure so in that folder there’s a you know you can see the agent folder you should drill into it there’s a slash config.xml file and in that slash dot config.xml file there is a what we call it the perimeter tag with that with the attribute name URL so in the one that you downloaded by default it sets to your local host so it’s local host call colon 8080 if that if that’s what you’re using okay just then you don’t have to do anything just just leave it be if it’s not say you’re like John you’re using a instance in the cloud then make sure that that you’d modify the URL value to http s okay and then followed by oops the instance name dot service – Norcom awesome okay so the next step once you have configured the config dot XML obviously just go ahead and use this start out start Sh script to start it up yeah you see that and then you’re ready to go one of the things that the first thing you want to check is now that the Miss service started up the question is you know did it actually connect to the instance that you expect right so the way you do that is by going to the mid server table and and you can check to see if you have a mr. automatic Lea register itself in the midst of a table so the the copy of the the mid server I gave you already has a name called local underscore mark underscore server one and you see if you don’t see it there then that means that there’s a problem from the mid server to communicate to the instance okay okay that’s fine that’s fine so do know there are a few things like that that we need to check to make sure so this is a good this is a good learning process so what could go wrong right so the number one I would check to see so in the mid server config XML the the username and password that allows the miss river to communicates into the instance for our purpose I used admin with the password admin so if that if that user doesn’t exist on your instance make sure that you go to your user table and make sure that admin admin credential is available so okay so if you’re if you’re conflict XML if it is actually if it the user name says miss server you have two choices either you can go to the instance and create a username miss server and Pat with the password also is the same as the username also miss server all lowercase one word or you can change it on the config.xml file to just admin admin admin and I apologize I think I got mixed up in the in the package that I gave to you guys I thought I thought I configured admin

admin in there one second yeah so in a password yeah you need to remove the encrypted and basically just do oops so the if you’re just curious the encrypt equals true that that just allows it to um so so like the like the way I’ve typed on my screen it’s admin admin but once the it connects to the instance successfully it’ll actually rewrite it into the encrypted form okay that’s fine if you if you’re having issues the the next step is um so I mean make sure that the tape the username on a instance correct and make sure your URLs correct if none of that work then the last thing you want to I mean uh either the next thing you could do is actually look at the log on the MS server so in the agent folder there’s a log log directory so in the in the log directory you’ll see you’ll see agent zero asian zero dialogue that’s zero so look at that to see what it tells you yeah so so if anytime there’s something wrong I would I would take a look at the mr. Pibb log just to see so now that you have your miss server up the next thing the next thing that we’re going to do is basically configure the credentials table well actually before we even do that I want to explain what we’ll explain what we’re going to do so the objective here is actually for you to make my computer talk okay so there are two things I need to give you the first thing I’m gonna give you is the credential to access my laptop okay and the second thing I’m gonna give you is my IP address so that you can you know where to target it okay so the first thing it was going to configure the credential table so go ahead and go to the credentials table under discovery yeah so go ahead and click new and then and then sort of follow my example so let me let me type into the chat what the username password should should be so the username is discover me the password is a disc zero okay so once you have that go ahead and save the record and it’s all it’s gone it’s gonna be automatically pushed to the MS server what once you do that so the miserable will have your misery we’ll have my quote at a credential to my laptop okay so the next thing if you haven’t realized it’s already is that in a in on a Mac you can actually make your Mac talk anytime right so for example I can make my Mac say anything second so for example I can type say hello world and then my laptop will say hello world so I want you to do the same same thing but to my computer and through the mid server okay so I’m gonna give you my IP address and before I give you my IP address I just want to let you know that this is my laptop and I work with this laptop so do not do anything to me okay please I mean I I think I think I locked it down to a point where it’s I think mostly read-only but you know I’m sure there’s places I didn’t lock it down so okay so I don’t have any grudge against you so you know don’t give me any reason to so that let me see my IP is my type in chat one second okay okay here’s my IP address ten 0.68

at 107 and I’ll walk you through what that what how did how to do have a constructive record right so go ahead and navigate to the ECC queue table so once you navigate it to the ECC queue table the very first thing is you want to do is specify which miss server you want to use so in your case it is mid that server dot local underscore Mac underscore server one okay then the next thing is topic it’s called SSH command okay here we go so in the topic I want you to type in a sage command it does mat that the casing does matter okay so it is uppercase SSH see and the rest of the lowercase the name is the the command you’re going to run so in this case you can do say hello world but I want you to be creative okay so surprise me what you’re gonna say the obviously the password is this is the password here so and I passed up the IP so IP address so I was looking at something else and saying it and then the Q is an output so let me know when you’re here basically this is you’re constructing a single unit of work for the MIS server to do right so wait a mr you wanted to execute SSH command we wanted to say hello world and then I used on an awesome somebody did it awesome how do you do bug it okay that’s a fair question so the very first well the very first thing that that’s that you will take a look at is obviously the the the ECC queue record that you created so the vet so in this case you will say you look at your EC s Q output to see if it’s been picked up right so you know so in this case say I create an out ECC Q output record for the miss over to pick up if it stays in ready state that means that your miss server never never actually processed it right so in this case you can see it’s processed so that means – server actually picked it up and it did something and it actually executes a command back that’s how you that’s how you how you would know if the miss if so if the output record stays ready stays as ready that means and the miss miss ER didn’t pick it up and the natural thing to check is to go to miss server table and to see if your mid server is actually up and running so in my case you would you would see I mean in this case you would see the status being up or down if it’s up then that’s that means that the Miss server is somehow well it’s it’s it’s not likely that it’s it’s up and not not actually picking picking anything up on the ECC queue so chances are it’s fine showing down then you have to go to the agent log on the Miss server to see what’s what’s wrong with it so the very first thing I’m going to do is I’m gonna change the password all right so I’m going to go ahead and change the password so if anything you want to execute now now as a time if you do run again just automatic sends it again all right I’m locking it down all right

let’s move back to our presentation and so first of all I want to apologize that you know if it’s a it’s a it’s chaotic and and I hope that that you know that you understand that II this is actually plugged the best way you can learn something right so like you know there’s chaos there’s things in trouble troubleshoot and and and you know you’re understanding more in terms of how everything works um I also learned a lot today just I’ve been thinking about how to what’s the best way to basically improve the set of experience because obviously there’s there’s issues with the credentials there’s obvious things like table to troubleshoot credentials what’s wrong with it and I think there’s a lot of things that can be improved upon so I thank you for that even just good for me to see that so I want to continue on and with the myth server and hopefully hopefully by now you understand what a other micellar works right so obviously in in in in our product you would never for any customer you never construct a single unit of work for the micellar to do you know it’s typically wrapped by discovery or orchestration and things like that right the but but you know as developers here what I really want you to understand all the takeaway here today is it’s for you to understand how the mystery really communicates with the instance and how you can mister there’s something to do okay and and how that process really works so continue on the mist server there are some other features that fought that’s worth mentioning the we have clustering capability as well it’s um it’s has low balance capability as failover it’s a it’s a fairly simple straightforward with clustering it’s not it’s nothing too fancy at the moment it could change in the future but for now that that fits our use case it’s also got things like capabilities and IP ranges you can specify on the new server meaning that and you can you can say if mr. is capable oops mr. is capable of performing SSH powershell OS SNMP so on and so forth and which IP range ranges it’s capable of of discovery because we get as you can imagine typically custom customers network they they probably have VLANs that probably don’t even talk to each other so typically what we tell our customers that hey you know for the network segments that do not talk to each other go ahead and put a miss server in each one of those networks so that you know you don’t get into a situation where the mid server is it’s talking say you know you installed my server in America and it’s talking to Asia you know going across the way and typically we don’t we don’t want things like that happen we want to miss over on their particular the particular network to discover the devices within that network and send a traffic back to the instance through their way in okay there’s additional things you can look at interest in the server there’s performance report and and resource usage that’s built into it you can see there’s additional customizable functionality such as you can develop JavaScript probes that allows you to extend what the beyond what the masseur can do today so for example you could drop you can drop in a jar file on to the mid server and you can write your javascript probe that basically interacts with the with a with a jar file do the rhino write exactly the same way you would do on the instance you know if you haven’t learned that already so on the instance you can write javascript and you can call certain functions or methods in Java through right now so it’s the same thing that happens on the mid server as well you can write a JavaScript probe that allows you to to to basically reach in to the jar file and interact with it so that allows customers to basically drop in a custom jar file date of their choice and then they can they can interact with it you know what – whatever third party so an example of that in terms of what we do is force a VMware integration we were capable of talking to VMware vCenter and being able to grab information such as you know what are the ESX boxes one

other the VMS running on it so we leverage a jar that’s provided but a third party and and basically that a lot we then ride the probe JavaScript probes to to talk to the vCenter and grab that information so I’ve known of a couple other customers who have done similar things with a mainframe to leverage the similar scheme to get that to get that done any questions okay so that’s pretty much it in summary once again if it is anything that at all that I want you to take away is really understand the the architecture here between the mid server and instance and what it’s used for and again the instance the brain the misura is the hand that reaches the customer Network and a ccq is a table that allows that communication to happen that’s pretty much it in terms of this session any any questions