Episode Transcript
Speaker 1 00:09 Hey listeners. Hey Ben. Hi Jason. How's it going? It's going well, man. How about yourself going pretty well? So, uh, listeners, we thought we would bring you a new series in the workflow show here today. What we're going to do is focus on, uh, meeting entertainment basics, and we're going to call this series eMoney basics, surprise. We are going to focus on five different aspects of media and entertainment basics. And these are sort of the high level, big ones that we talk about when we spec out a solution or start talking about a solution and those are ingest storage, work in progress, delivery and archive. So these are kind of like very large areas that we make sure we hit when we engage with, uh, with any client on a solution, whether that be just a storage solution or a ma'am, these are all things that we want to be talking about and thinking about as the environment will, uh, you know, will be built.
Speaker 1 01:04 So today we are going to start with ingest. So that is our first episode in this five episode series. Perfect. So ingest what we're talking about with ingest Ben, right. Is where is media coming from and where's it going to, I mean, that's, you've got it. Yeah, that's, that's the pretty big question. And right now we could be talking about ingest to two storage from a camera, you know, ingest to, uh, ma'am from, uh, San, uh, many different ways we could describe that, but, uh, we're gonna, we're gonna kinda kinda touch on a little bit on all of them. We want to keep this kind of short because again, it's a five episode series and we wanna, you know, get to everything here. So Ben, let's talk about where could media be coming from?
Speaker 2 01:49 Yeah, so the obvious first choice is from camera cards or specifically from cameras, right? Um, we've got our high quality cameras coming in from the field. Sometimes they're SLR cameras, sometimes there's purpose built for video and we're ingesting files sometimes directly off of those and SD cards or some sort of, uh, flash memory cards that we then bring into our computer and transfer to a hard drive and then hopefully ingest into some other medium of storage. And one of the things that I think is really important to think about in any ingest process, but specifically from camera cards, because you want to reuse those again, is making sure that there's some sort of a check sum happening when you're pulling those files off of that card. And what we mean by that is that there is a piece of software between the media card and the storage device, or that references both the original media card and the storage device, making sure that they are bit for bit accurate so that there are no discrepancies. You didn't miss a one or a zero. And that each file is, are 100% identical so that you can then go ahead erase that camera card and put it back into circulation, knowing that you've got everything correctly and there's a number of cards or they're being reused all the time. It's not like a tape that's going to go on a shelf or something like that. Right, right. Yeah, exactly. There's a number of good software packages out there. Um, I think ShotPut, pro's the one that always comes up on top of my mind.
Speaker 2 03:31 So you get it. That's, that's one of the primary acquisition formats that we see coming in. Others include what Jason likes the
Speaker 1 03:43 Live and just from, um, something like, uh, a remote server or, uh, could be, um, I mean, I guess we could be talking about for live and just, we could be talking about a broadband IP, I'm sorry, a broadband live and just, we could also be talking about an IP based, uh, live and just,
Speaker 2 04:00 Right. So if you're streaming from on location, as I'm sure some of us do now, and many of us will do in the future and that's going through our various tubes on the interwebs up to the servers in somebody else's server room that we call the cloud and then back to hopefully our local storage, we've got to make sure that that data stays accurate and that it's usually taking some kind of a wonderful content distribution network back road, like Akamai or something like that. So that by the time it gets to your storage or cloud storage, that you're paying someone for that it's in great shape.
Speaker 1 04:39 Gotcha. Okay. So moving right along. What about other sorts of, we talked about camera cards and live and just, uh, what about just like something we downloaded from the web, like stock media, or maybe even something that we already had on our storage, like legacy media that might be SD or maybe even captured from a tape or something like that.
Speaker 2 05:00 Yeah. Right. So all of those shelves and shelves worth of HD cam and DV tapes that everybody is ignoring and that's gathering dust in your corner, or maybe it's even hard
Speaker 1 05:14 Of stuff that we worked on in years past.
Speaker 2 05:17 Right. I'm sure many of you have stacks and stacks of old G-Tech drives sitting somewhere in a closet that keep you up at night, uh, that perhaps you should spin up and make sure that the, uh, electrons still align and that the magnetics are still working and that your files are still alive. So yeah. Word to the wise, please, dear God, spend those hard drives up just to make sure that everything stays moving, that the ball bearings don't get crusty and that, um, you can still get back to that stuff and that it doesn't die a terrible death via neglect.
Speaker 1 05:49 And just to spend like another 10, 15 seconds on that, it really is just literally spinning those drives up and making sure that the contents there and that's it right. We
Speaker 2 05:58 Don't have to like spray them with WD 40 or anything. Like, please, please, don't spray your hard drives with WD for me please. That's right. Just, just spin them up, make sure they still work, let them spin for 10 minutes and then put them back in the closet. Uh, do that once a year, every six months, just make sure that you're doing that so that things don't sit neglected for years because that's when they, they die. And then you try and go back and say, Oh yeah, I had that on this hard drive. You've dutifully labeled with a date and a project name on the exterior of the drive. And then when you go it's dead and then everybody cries don't do that. Okay. So, I mean, other than that, it's just kind of the miscellaneous stuff. Like, did we get the media from somewhere else?
Speaker 2 06:38 Like a client or maybe, yeah. I don't know. Something that somebody just handed you and say, Hey, we need to use this in the next edit. It's kind of falls into that category. What about some places that we could ingest to? Uh, so, so what are some considerations there? So I would think, I would think that would kind of be based on how will the media be used, what are we doing next with it? Uh, yeah. So that'll, I think that gets into a little bit more of another episode that we're talking about, um, in and around storage, but because we're talking about ingest, obviously the next step in the media supply chain is a shared storage volume. If you're working as a group, um, or if you're just working as a freelance editor or something like that, then it's probably coming into your hard drive on your MacBook or your, whatever your edit workstation is.
Speaker 2 07:29 I'm a Mac guy. So my brain always goes to Mac, but it may be a wicked PC. Maybe you built it yourself, who knows. Um, but clearly the next step is from storage to storage and there that's where we get the chance to organize things. Okay. So we might be using a, an ingest tool, like maybe say prelude or something like that to pull this media off of cards or something like that. And that's kind of where we have the opportunity to say like, are we going to organize this into folders on the file system, by like a shoot date or maybe a title? Is that what we're talking about? Yeah, you got it. So there are a number of really useful tools to help you both with generating useful naming convention because we're huge sticklers for that. That's always really important. Um, largely in part, because we want to be able to find things again, fairly quickly, always a big fan of date based naming conventions. And so if you start with a year, first month, second day last, then when you're searching for it in the file system, it shows an order according to date, and that's an easy way to trace things down. Let's actually, let's actually say that one more time, because I think a lot of people, I think a lot of people do the traditional, like when we look at a date here in the U S we see month. Oh yeah, yeah. So a lot of, I think a lot of people end
Speaker 1 08:56 Up naming their files and folders with that naming convention. When you put the year first, the files will always sort alphabetically by year, month day, which is, that's kind of what you want. Right. You don't have to worry about, you know, what order they're in or anything like that. It just kind of sorts the way you want to see it. Right. Yep. So,
Speaker 2 09:15 Yeah. And then what I've always done is, um, or at least I won't say I've always done because it took until I started really getting into the it stuff that I started realizing that naming conventions matter in file systems, you know, as a creative audio guy, myself, for years, you just want to get the project going. And so you'll use the default name is project. Maybe you'll name the folder, something else like, uh, the project, like the band's name or the project's name or whoever the client's name is.
Speaker 1 09:49 And hopefully you won't include the words new or final in that naming convention
Speaker 2 09:53 Or final final version. Yes. This one this time. Yes.
Speaker 1 09:58 Uh, okay. So, okay.
Speaker 2 09:59 So back yeah. Naming conventions. So we could do all that
Speaker 1 10:03 Episode on naming conventions by the way. And we probably will, because this is a, I would say an area of frustration for many of us in the industry is the whole idea that, uh, that, that we want to be creative. So we should be creative with file names and folders too. Right. That's kind of emojis. We sit to use emojis. That sounds like a great idea. Right? That's not such a great idea. Yeah, please don't do it, but we'll, we'll get into it at some point with, uh, with a future episode. Um, so we, we might also do some organization like placing things into some sort of a hierarchy based on our project or based on maybe the date of things as they're being ingested into our infrastructure system. Yeah.
Speaker 2 10:45 I think one of the good things to note with that is within the operating system, there are specific conventions like underscores or hyphens that are a great way to separate specific pieces of data within your naming convention. Right? Because those are easy to interpret later on say, if you wanted to ingest things into a man, which I'm sure we'll talk about very soon. But if, if in your naming convention was, we're talking about naming conventions, now you use underscores or hyphens. Those are easy to extrapolate later. If you've got a date, if you've got some sort of brief VAT project name, or maybe there is a project numbers or something like that. Yeah, exactly. And what we can do later is then hopefully expand that into human readable metadata. That then becomes something that you can search on. Obviously, if it's just in your file system, on your Mac, within the finder, on your sand and your Nass, you can search those specific elements that way as well. But if you have a ma'am, you can search for more than one criteria so that you can find things more readily.
Speaker 1 12:10 So you mentioned files, I'm going to get to ma'ams in a minute, but you mentioned a file systems there. So let's talk very briefly about storage. Cause we are going to talk about this, uh, the next episode in this series, but storage, we could be talking about your sand, your NAS, you already mentioned naming conventions. That's a huge thing. Um, what about cloud storage? I mean, we can also ingest the cloud storage, right?
Speaker 2 12:34 We can for sure. It's a little bit, so there's a couple of different tiers of cloud storage, right? There is kind of the enterprise class object storage, all, uh, AWS S three or Backblaze B two or Microsoft Azure or Google cloud. IBM cloud IBM cloud indeed. Um, that is really that you're not going to necessarily easily get access to, unless you're hooking something up like a ma'am in front of it or something like, I don't know, cyber Docker, something that can push and pull things back and forth to those specific buckets, which I don't know why they call them buckets. I'm sure there's a good reason. And I'm sure maybe we'll talk about that in another episode. Sure. Um, yeah. Uh, it's really built for it use versus stuff like Dropbox or box.com or any of these other, um, cloud native tiers that are, that have really nice, easily usable gooeys that are probably just built on those backends of enterprise cloud.
Speaker 1 13:48 Yeah. You know, using box or Dropbox or one of those one drive, one of those kinds of services can be a little complex just because of the user structure and the access and all that kind of stuff that has to be set up to get that running. I mean, typically if you're just kind of sending one thing to a sand or something like that, it's, there's this whole order of how do we set that up? Is the receiving end acting as a user? You know, how do we do that? So that can be a little weird, but, um, that is, you know, completely viable way of getting content to cloud storage. And maybe even from one user through cloud storage to another on-prem pile of storage, like a file system.
Speaker 2 14:26 Yeah, exactly. There are some really good file systems out there. And in fact, a lot of really good companies these days are building that tiering functionality into their storage file systems. So that if you have a cloud based tier that you might be say, you're in the field, you've got a ma'am tear. Maybe you're using something like cat TV or, um, and you're ingesting from the field directly into cloud storage. And then maybe there are rules in place that says this specific bucket will always get, you know, copied back down onto the sand. And so that our friends who might still be in the office waiting for the files, waiting to get to the edit as fast as humanly possible, get access to those files as fast as humanly possible, that of course relies on wicked fast internet pipes. So wherever you are, hopefully the speed with which you're uploading gigabytes worth of files into the cloud and that the internet gods are with you and you have sufficient bandwidth.
Speaker 1 15:35 Absolutely. All right. Well, let's, let's move on from this part and talk about ingest to a ma'am. Um, and I think for our purposes today, we're, we're going to sort of cover ma'am and workflow orchestration in the same discussion because they can be separate. They can be the same, the same system, same environment. But when we talk about ingest to a ma'am, I have to say as an engineer who deploys lots of ma'ams, uh, this is usually one of the most complex discussions that we have, uh, when implementing and deploying a new ma'am, which is how should the ingest process be handled? How do we bring content into this ma'am? And, uh, it's, it's really because every platform is different and every client has different needs. They want different things to happen. They want the robots to do different things when, when the media is brought into that system.
Speaker 1 16:29 So, uh, there are moms out there that are really, really, really capable and have lots of really awesome, like flexible, you know, workflow orchestration, but they don't have any built in and just functionality. You have to build that all, you know, during the implementation of the ma'am, there are also moms that have a sort of like vanilla ingest process where, you know, you, you bring the content in. Usually we find that most clients need a little bit more than that. They might need a lot more than that. They might need, right. Proxies to be generated or, uh, for the files to be renamed, or maybe even move to a different part of their storage. That's locked down from, from, you know, red Xs by users. Um, so when we talk about ingest to a man, we can get really complex really quickly again, because of that fact that everything is different, every platform and every client has a different need. Um, so, uh, yeah, I mean, I w what are some other considerations we have for ingest to a ma'am?
Speaker 2 17:25 So this coincides with the file system as well, right? There are two kinds of, well, when I think about the ingest process, as it gets associated with ma'am, I think of four defining properties, one, will it be manual or will it be automated? And then will it have a brain fart? Forget what it was going to say? Yes, it will.
Speaker 0 17:56 <inaudible>
Speaker 2 18:10 Eh, sing me a song when I am scared and can't sleep versus, um, Oh, obfuscating the file system. Right? So the other main point of concession or consideration when we're designing a ma'am ingest workflow is do your trust, your end users, do you want to hide the files and folders from them because you think they're going to do foolish things in the background and move things around, and then you can't find them later in the ma'am and everybody's unhappy, or are you going to have a little bit more of an open file system where you've already got good, um, naming conventions, you already have good templates and file system hierarchies that then you're ingesting in place so that you, outside of the ma'am, where there is a workflow that people might be accustomed to already. Um, and this is where Jason, I'm sure we'll talk about change management, where, um, we don't want to design an ingest workflow that nobody will use, right?
Speaker 2 19:22 We want people to be as comfortable as possible if there's anything within the organization that works well already. If they've got good naming conventions, if they have a good file system hierarchy, um, if they've been using tools like prelude or, um, digital rebellions, post haste, which is a really fantastic way to templatize your folder structure and on new project template, every time you need it, then we don't want to take that away from anybody because it works right. And we're here to enable them to work better and faster. We don't want to give them extra steps. Well, we will, but we want to make those extra steps meaningful. And to make you want to do those extra steps, because you'll get a hell of a lot more out of it later on. So potentially adding a few little steps that are way less painful than the steps that were previously there. Yeah, exactly. Like you may still do some of those steps where you're using a template, but maybe you're just filling out a metadata form. And then maybe it's part of something like, um, cat DVS, new project creation tool that then spawns those file system folders for you, and then creates a catalog or a collection inside of the ma'am and populates it with some base metadata. And maybe even
Speaker 1 20:48 Your project file based on naming conventions and gives it a new number or a new code based on, um, an increasing, uh, numerical structure. Kilburg you're blowing my mind right now. We can do all this with a ma'am. We can. Yeah. So what we're really kind of talking about again, is the fact that this is, this is really illustrative of the fact that ingest to a man can be very complex. There, there can be a lot of automation involved. One thing we talked about was the naming conventions, for example, just the process of harvesting out parts of the either file path or just file name. And then saying, when we see this three letter code, we want to translate that into a metadata field as this human readable text, you know, that this is all possible, it's all doable, but it's one of those things that needs to be developed.
Speaker 1 21:41 And, uh, first of all, decided, you know, and then developed, uh, and then, and then tested, tested fervently if I'm using that correctly, right. Rigorous testing rigorously as the word I was actually looking for. Yeah. Thank you. There you go. Right. Cause we want to make sure that people are going to use it because if you've spent thousands and thousands, maybe hundreds of thousands of dollars on software and servers to help you organize your media, get control of your media supply chain and speed it up. You better use that, or else you've just built a rocket ship that goes nowhere. Absolutely. Yeah. We also might be harvesting metadata from files like EXIF metadata, or XMP metadata or something like that, that could be embedded in those files to use for metadata in the ma'am or even for triggering workflows or something like that. So, uh, lots of things there.
Speaker 1 22:37 Um, other automations that we often talk about and adjust almost always, we're talking about proxy generation, especially with cloud-based ma'ams because you're not going to stream the high resource or even the mezzanine really over a basic internet connection. You're going to want to stream something in an MP for container that's got the right streaming flag set and all that jazz. Um, another thing, another automation that we often build into ingest workflows is archive on ingest, which we will talk about in the last episode of this series. Uh, the idea of archive on injustice that everything that gets ingested to your ma'am, uh, is immediately archived and not necessarily purged from the storage. So thereby you've sort of eliminated that task, uh, that often takes a lot of time of sending something to an archive. You've sort of combined that with the whole process of bringing it into your environment in the first place, uh, and there's there's pros and cons to that whole workflow, but that was just another thing that we wanted to mention. So maybe part of that archival and ingest workflow, maybe we're making a low risk proxy and also a mezzanine copy. Maybe it's just pro Rez and then your
Speaker 2 23:45 Highest resolution files that you might not edit with, but they're worth all of the money in the world because they are what was recorded on site. They're the best known copy of it. And they get saved because maybe there's compliance issues that for insurance reasons, you need to make sure that there are two copies. And so that does that also the added benefit of making the archive is just a deletion. You don't have to go ahead and archive the file later. All you have to do is hit archive in the file, says, Oh, okay, I'm going to race this because I know it's safe over here already. Right?
Speaker 1 24:24 Well, I think we have, I think we've kind of reached time on this particular episode in the series. And I want to just do a quick recap. So what are some really high level considerations that we talk about when we talk about ingest? Oh, you're asking me. Yeah. I guess it's funny. Cause we have no guests. We really could just, we really could just probably rattle these off right. Then like of course, yeah. Naming conventions. Uh, well, let's start with the overall question, which is where is the media coming from and where is it going to, how are we going to name any files that are created as a result of that process, understanding the difference between ingesting to storage and ingesting to a man because those, you know, those they're both ingest. Those can be a little confusing. Sometimes. Sometimes we do have to say, Oh, I'm just talking about ingest to the storage, not the man. Right, right. And you know, also just the whole idea that we really need to define and adhere to a repeatable process that everyone knows. Hallelujah. Right. So that's the, those are the big takeaways. And I just want to tell a really quick, funny story about ingest. I remember the first time in the industry that I'd ever heard the word ingest, and I kind of laughed at myself cause I was like, what are we doing? Eating the media.
Speaker 1 25:38 So anyway with that one, all right guys, uh, that was, uh, that was our ingest portion of this five episode series on media and entertainment basics. We will hook up with you guys here and, uh, maybe a week or two on the next episode in this series, which will be on storage. The workflow show is a production of Chesapeake systems. The show is co-produced by Sierra O'Connor and Sarah Schachner I'm Jason Wetstone senior workflow engineer, and I'm Ben Kilburg senior solutions architect, and Ben also expertly records and edits the show. If you enjoy the show, please subscribe in your podcasting app of choice. And why not tell a friend, a coworker or a client about the show. We'd love to hear what you love about the show too. So email us at podcast at <inaudible> dot com. Thanks for listening to the work flow show.
Speaker 0 26:30 <inaudible>.
