Editing Creating Custom Lipsync Code in Flash (section)

===FOIST===
For my lipsync process, I've personally settled on using eight basic mouth shapes:
* ''static closed mouth''
* ''mm''
* ''oo''
* ''ee''
* ''eh''
* ''nn''
* ''ah''
&
* ''oh.''

On ''"mm"'', I squish the head down a little, & on ''"oo"'', I squish the head down a little more. ''"Static closed mouth"'', ''"ee"'', ''"eh"'' & ''"nn"'' get no squash or stretch. Then, on ''"ah"'', I stretch the head up a little, & on ''"oh"'', I stretch the head the most. I think it makes for a really fun cartoon-y Humongous Entertainment-esque result of the character being more alive, in spite of the many limitations in play.

I followed an example that I considered to be most convenient at the time, which I can't find the origin of anymore. It was an AS2 example which I then revamped to work in AS3. It looks like this:

[[File:lipsynctool1.png|500px]]

How it works is you load in a sound file, hit Play, then press a button as you listen to the sound file, and every time you press that button, it will output the timecode of the press, along with a command afterwards to change the mouth graphic to a different frame. So you get a script like this:

[[File:lipsynctool2.png|500px]]

The original script had the script respond to mouse clicks, but I think that one can get rather tired of clicking faster than tapping the keyboard, so I changed my version of the lipsync tool to being activated with the space bar.

What I ended up with goes a lil' something like this: you make a button (mine is an ugly radial black-green gradient circle, as it was when I found the example) and call its instance name ''"play_btn"''. Then you make ''another'' button (ugly radial black-red gradient circle) and call its instance name ''"stop_btn"''. Then, you make three little dynamic textboxes for the timecode information you'll be needing to know and manipulate-- call the instance names for these three textboxes ''"startatBox"'', ''"currentBox"'', & ''"durationBox"''.

Then, right upon the frame these buttons and boxes reside, you put this ActionScript:

 <nowiki>stop();

// REFRESH LIPSYNC VARIABLES

for(var a:int=0; a<1000; a++) {
	this["doOnce" + a] = 0;
}

// BUTTONS

play_btn.addEventListener(MouseEvent.CLICK, startSound);
stop_btn.addEventListener(MouseEvent.CLICK, stopSound);

// IMPORTING

var SyncSound:Sound = new joe014();
var SyncChannel:SoundChannel = new SoundChannel();

// DURATION

var duration:Number = Math.round(SyncSound.length)/1000;
durationBox.text = String(duration);

// START BUTTON

function startSound(e:MouseEvent):void {
	SyncChannel.stop();
	for(var a:int=0; a<1000; a++)
    {
        this["doOnce" + a] = 0;
    }
	var startat = Number(startatBox.text) * 1000;
	SyncChannel = SyncSound.play(startat);
}

// STOP BUTTON

function stopSound(e:MouseEvent):void {
	SyncChannel.stop();
}

// CREATING NEW LIPSYNC SCRIPT

var n:Number = 0;
var currentTime:Number;
stage.addEventListener(KeyboardEvent.KEY_DOWN, myKeyDown);
function myKeyDown (e:KeyboardEvent):void {
if (e.keyCode == Keyboard.SPACE) {
	if (currentTime>0)
	{
		n = n+2;
		var stopCheck = "this.doOnce"+n;
		trace ("if (currentTime > "+currentTime+" && "+stopCheck+"!=1){"+stopCheck+" = 1;MovieClip(root).joeava.joehead.gotoAndPlay(2);}");
	}
}
}

// CONSTANTLY UPDATING SCRIPTS

stage.addEventListener(Event.ENTER_FRAME, EnterFrameLoop);
stage.addEventListener(Event.ENTER_FRAME, DialogueScript);

function EnterFrameLoop(e:Event):void {
	currentTime = Math.round(SyncChannel.position)/1000;
	currentBox.text = String(currentTime);
}

function DialogueScript(e:Event):void {
	currentTime = Math.round(SyncChannel.position)/1000;
}</nowiki>

One of the important things I've placed down twice in the code is to completely refresh those  <code>doOnce</code> variables that are being generated that ensure that no mouth shape plays more than once. Obviously, in theory, the mouth commands for a single line could go HIGHER than 1,000, but... what kind of good game would ever have a ''single line of dialogue'' that needed more than 1,000 mouth shapes? I think it's a good limitation for game-feel.

Another important note-- obviously, your own command that gets output would be different, depending on the instance names you chose and your symbol hierarchy in place.

For me, as is seen in my code example, my character is in a main symbol with an instance name of ''"joeava"''. Inside it, the head is in its own symbol with an instance name of ''"joehead"''. Then, in there, the first frame of the head has the ActionScript code, <code>stop();</code> to keep all the mouth frames from being on a constant loop. After the <code>stop();</code> frame are groups of three frames for each phoneme. The frame labels are set to each of the phonemes mentioned above, then on the third frame of each of those mouth shapes, I turn that frame into a keyframe and enter the code, <code>gotoAndPlay(1);</code> to bring it back to the static closed mouth. And, of course, your dialogue sound clip will, in all likehood, not be called ''"joe014"'', so just replace the common-sense parts with the stuff that would relate to your situation.

... To reemphasize, ''MY'' commands are targetting ''"joehead"'' in ''"joeava"'', and asking ''"joehead"'' in ''"joeava"'' to play a selected phoneme at a selected time. You can cue ANY symbol of ANY instance name any WHERE to play or do just any THING! It's all up to You.

Then, after I'm finished hitting Space bar to the beat of the full piece of dialogue (or starting that dialogue at a specific part by entering in a timecode in my ''"startatBox"''), I would look through my outputted code, and then fill in the appropriate actual mouth shapes to correspond with each line of code-- so instead of a frame reading...

 <nowiki>if (currentTime > 0.887 && this.doOnce6!=1) {
	this.doOnce6 = 1;
	MovieClip(root).joeava.joehead.gotoAndStop(2);
}</nowiki>

... it'll read...

  <nowiki>if (currentTime > 0.887 && this.doOnce6!=1) {
	this.doOnce6 = 1;
	MovieClip(root).joeava.joehead.gotoAndStop("oo");
}</nowiki>

Then I paste the output into the open DialogueScript in my code, then hit Play and see how the line is looking. If I timed anything wrong, I'd modify accordingly. This might seem a little taxing, but especially on longer dialogue, this line-by-line approach actually tended to be WAY faster than doing it all frame-by-frame, oddly enough.

After all that finagling, to implement the lipsync in my game, I have a little Movie Clip hidden off on the side of the screen called ''"dialogue"''. Inside of it, the first frame is called ''"none"'', where it obviously stays any time no dialogue is played. Each frame after that represents one of the dialogue lines waiting to be activated by an in-game interaction. So, when something on screen is pressed, and the code says something like...

 <nowiki>dialogue.gotoAndStop("joe014");</nowiki>

Then the code on the ''"joe14"'' frame is activated, which I shall show the most relevant bits of (some of the sound-related stuff DOES depend on what I put before it using [https://github.com/greensock/GreenSock-AS3/ GreenSock] audio stuff, but I think it's pretty readable):

 <nowiki>stop();

// SKIP

stage.addEventListener(KeyboardEvent.KEY_DOWN, skipdialogue_joe014);

function skipdialogue_joe014 (e:KeyboardEvent):void {
if (e.keyCode == Keyboard.SPACE || e.keyCode == Keyboard.ESCAPE) {
	SyncChannel.stop();
	stage.removeEventListener(KeyboardEvent.KEY_DOWN, skipdialogue_joe014);
	stage.removeEventListener(Event.ENTER_FRAME, dialoguescript_joe014);
	gotoAndStop("none");
}
}

// REFRESH

for(a=0; a<1000; a++) {
	this["doOnce" + a] = 0;
}

// SOUND SET-UP

SyncTransform.volume = SyncControl;
var Sync_joe014:Sound = new joe014();
SyncChannel.stop();
SyncChannel = Sync_joe014.play(0, 1, SyncTransform);
SyncChannel.addEventListener(Event.SOUND_COMPLETE, soundFinish_joe014);
function soundFinish_joe014($evt:Event):void {
    SyncChannel.removeEventListener(Event.SOUND_COMPLETE, soundFinish_joe014);
	stage.removeEventListener(KeyboardEvent.KEY_DOWN, skipdialogue_joe014);
	stage.removeEventListener(Event.ENTER_FRAME, dialoguescript_joe014);
	gotoAndStop("none");
}

// LIPSYNC
	
stage.addEventListener(Event.ENTER_FRAME, dialoguescript_joe014);
function dialoguescript_joe014(e:Event):void {
	currentTime = SyncChannel.position/1000;
	if (currentTime>0.046 && this.doOnce46 != 1)
	{
		this.doOnce46 = 1;
		MovieClip(root).joeava.joehead.gotoAndPlay("oo");
	}
	if (currentTime>0.139 && this.doOnce48 != 1)
	{
		this.doOnce48 = 1;
		MovieClip(root).joeava.joehead.gotoAndPlay("ee");
	}
	if (currentTime>0.464 && this.doOnce50 != 1)
	{
		this.doOnce50 = 1;
		MovieClip(root).joeava.joehead.gotoAndPlay("ah");
	}
// etc., etc., etc.
}</nowiki>

Haha, you'll see this particular code starts later than <code>doOnce1</code> -- that happened a lot at the time, due to me having to try multiple times from multiple points in the line of dialogue to get the timing right with my spacebar, leading to me modifying the code the second time around to start after the initial <code>doOnce</code> series had ended. <code>doOnce</code>? More, like... <code>doManyTimes</code>... ?